Risk adjustment in health care markets: concepts and applications Randall P. Ellis. Boston University and DxCG, Inc. Slides prepared for the Risk Adjustment Network (RAN) meeting in Dublin Ireland, March 6-8, 2008. Material is based on Ellis (2008), Jiang, Ellis, Kuo (2007), and Ellis and Mookim (2008). Page 1 Randall P. Ellis March 7, 2008
Outline of talk 1. Introduction and definitions 2. Conventional versus optimal risk adjustment 3. Empirical issues 4. Conclusions (Omitting discussion of predictive power and of International comparisons) (ten minutes) 5. Jiang, Ellis and Kuo (2007) on selection in US (3 slides) 6. DxCG classification update (5 slides) 7. Quantifying the overfitting problem (2 slides) Page 2 Randall P. Ellis March 7, 2008
Figure 1: Different terminology used related to risk adjustment Predictive modeling Risk Adjustment Case identification Severity adjustment Health-based payment Conventional risk adjustment Optimal risk adjustment Needs-adjusted payment Use any information available Prediction models Use health status information only Payment models Use health status, socioeconomic, and access information Ellis, 2008 Page 3 Randall P. Ellis March 7, 2008
Four agents and five primary contracting relationships Sponsor Plan payment RA Health Plans B Vouchers and Health Savings Accounts RA A C D RA Provider Contracting E Consumers Providers Ellis, 2008 Page 4 Randall P. Ellis March 7, 2008
Two different questions commonly asked What is the predicted resource use for an individual? Predictive modeling What is the best payment formula to use for capitated payment? Conventional risk adjustment (Van de Ven and Ellis, 2000) Optimal risk adjustment (Glazer and McGuire, 2000). Page 5 Randall P. Ellis March 7, 2008
Predictive models used for many diverse purposes in US measuring quality case management disease management high cost case identification underwriting plan selection by employers provider profiling plan payment by sponsor ( risk adjustment ) Page 6 Randall P. Ellis March 7, 2008
Models of Optimal Risk Adjustment Goal is overview of multiple papers, not presentation of one Primary topics: perfect versus imperfect information, imperfect signals available to the regulator, the incentive problems that health based-payments are intended to correct, the strategic responses to risk adjusted payments. Graphical approach Page 7 Randall P. Ellis March 7, 2008
Simplified 2X2X2X2 model two states of the world: Healthy and Diabetic two types of health care goods: GP and SP two types of consumers: Low and high risk types two possible signals about consumer types, S = 0 or 1 The only information that is contractible, and can be used for paying capitated providers are the signals about consumer types. Realized states of the world, actual levels of spending on each type, and true patient types are assumed not contractible. Page 8 Randall P. Ellis March 7, 2008
Simplest assumptions Low risk types are always Healthy High risk types are always Diabetic Healthy use only general practitioner services (GP) and cost α per year. Diabetic consumers use both GP and specialists services (SP) and cost α+β. Hence α is the annual cost of Healthy consumers while β is the incremental cost of Diabetes. Half of population is low risk (Healthy) half is high risk (Diabetic) Page 9 Randall P. Ellis March 7, 2008
Spending on Specialists α + β Figure 2: No Risk Adjustment when quantities of each service supplied are exogenous. α+ β/2 B Diabetic (high risk) α X A Healthy (low risk) α α+β/2 α+β Spending on GPs Page 10 Randall P. Ellis March 7, 2008
Spending on Specialists α + β Figure 3: Perfect and imperfect conventional risk adjustment. α Healthy A * B B X A A α B * Diabetic α+β Spending on GPs Page 11 Randall P. Ellis March 7, 2008
Spending on Specialists Figure 4: Capitated quantities of services (A 1, B 1 ) will differ from quantities offered under Fee-For-Service (A 0, B 0 ) due to supply-side moral hazard. B 0 Diabetic B 1 A 1 A 0 Healthy α 1 α 0 α 1 +β 1 α 0 +β 0 Spending on GPs Page 12 Randall P. Ellis March 7, 2008
Spending on Specialists Figure 5: Conventional Risk Adjustment with quantities of services offered strategically determined. B 0 Diabetic B 2 A 0 α 0 α 2 α 2 +β 2 α 0 +β 0 A 2 Healthy Spending on GPs Page 13 Randall P. Ellis March 7, 2008
Glazer and McGuire: Optimal Risk Adjustment Choose payment weights so as to maximize some objective function. Need not pay expected cost for each signal. Current costs need not reflect optimal choices Van de Ven and Ellis: Distinction between acceptable costs and actual costs Objective function depends on the problems to be fixed Page 14 Randall P. Ellis March 7, 2008
Spending on Specialists Figure 6: Optimal Risk Adjustment with quantities of services strategically determined. B 3 B 0 Diabetic B 2 A 3 A 0 A 2 Healthy α 3 α 0 α 2 α 3 +β 3 α 0 +β 0 Spending on GPs Page 15 Randall P. Ellis March 7, 2008
Spending on Specialists Figure 7: Risk Adjustment with taste or income heterogeneity. B B Diabetic A A α Healthy α+β Spending on GPs Page 16 Randall P. Ellis March 7, 2008
A few empirical observations Table 2: Overview of major US claims-based risk adjustment models Rate cell or linear regression? Age/ inpatient all Acronym Label Early Reference gender diag. diagnoses CI Charleson Index Charleson et al., 1987 reg X X DCG Diagnostic Cost Groups Ash et al, 1989 reg X X ACG Adjusted Clinical Groups Weiner et al, 1991. cell X X X CDS Chronic Disease Scores Von Korff et al., 1992 reg X X HCC Hierarchical Condition Categories Ellis et al, 1996 reg X X X pharmacy information procedure codes CDPS Chronic Illness and Disability Payment System Kronick et al, 1996 reg X X X GRAM Global Risk Assessment Model Hornbrook, et al, 1996 reg X X X CD-RISC Clinically Detailed Risk Indication System for Cost Carter et al, 1997 reg X X X CRG Clinically Related Groups Averill et al, 1999 cell X X X X X ERG Episode Risk Groups Symmetry Health Data Systems, Inc. 2001 cell X X X X X RxGroups RxGroups Zhao et al, 2001 reg X X RxRisk RxRisk Fishman et al, 2003 reg X X Page 17 Randall P. Ellis March 7, 2008
Table 3: Predictive power of various information sets and various models Dependent variable is 1997 US Medicare total covered charges Square Root model (heteroskedasticitycorrected) GLM with link = log, dist = normal Two part linear Weighted OLS OLS model Partial Year Eligibles included? Yes No No No No Sample Mean 6,886 5,063 5,063 5,063 5,063 Number of Observations 1,380,863 1,273,471 1,273,471 1,273,471 1,273,471 R 2 R 2 R 2 R 2 R 2 Age and gender only 0.011 0.010 0.009 0.010 0.010 Prior year total covered charges* 0.089 0.096 0.113 0.120 0.105 Diagnoses organized by DCG/HCC* 0.104 0.108 0.103 0.107 0.105 Covered charges by DCG/HCC* 0.099 0.107 0.103 0.105 0.095 Covered charges by Place of Service* 0.140 0.145 0.136 0.145 0.126 Covered charges by Physician Specialty* 0.142 0.152 0.143 0.152 0.131 Covered charges by Type of Service* 0.150 0.155 0.146 0.154 0.134 All of the above except diagnoses* 0.154 0.160 0.151 0.160 0.138 "Kitchen sink": All of the above* 0.169 0.171 0.161 0.169 0.147 *All Regressions included a constant and 21 age-gender dummy variables Source: Ellis and McGuire, 2006, Table 1. Page 18 Randall P. Ellis March 7, 2008
Predictive Power of Various Information Sets US Commercially insured sample, prospective model Estimation Method Weighted LS OLS Two-Part Linear Model Partial Year Eligibles included? Yes No No Sample Mean 3560 3463 3463 Number of Observations 5,298,819 4,688,097 4,688,097 Information Set Rsquare Rsquare Rsquare Age and Gender only 0.0266 0.0293 0.0277 Prior Year total covered charges 0.0982 0.1027 0.0992 Simple HCC 0.1692 0.1746 0.1749 Covered charges by Place of Service 0.1894 0.2042 0.2055 Covered charges by Physician Specialty 0.1779 0.1924 0.1938 Covered charges by Type of Services 0.1977 0.2107 0.2036 Jiang, Ellis, and Kuo 2007 Page 19 Randall P. Ellis March 7, 2008
Figure 10: US privately-insured health care spending, by age and by gender, $12,000 2004 MEDSTAT Marketscan data (N=14.6 million) $10,000 Covered health spending, 2004 $8,000 $6,000 $4,000 Male Female $2,000 $- 0 10 20 30 40 50 60 70 80 90 100 Age Page 20 Randall P. Ellis March 7, 2008
Figure 11: US privately-insured health care spending by age, by health plan type 12000 2004 MEDSTAT Marketscan data (N=13.0 million) 10000 8000 Health care spending 6000 4000 Comprehensive HMO POS PPO POS with Capitation 2000 0 0 10 20 30 40 50 60 Age Page 21 Randall P. Ellis March 7, 2008
Figure 12: Risk-adjusted US privately-insured health care spending, by age and by health plan type, 2004 MEDSTAT Marketscan data (N=13.0 million) 12000 10000 8000 Health care spending 6000 4000 Comprehensive HMO POS PPO POS with Capitation 2000 0 0 10 20 30 40 50 60 Age Page 22 Randall P. Ellis March 7, 2008
Conclusions of Ellis (2008) Dramatic growth in interest in risk adjustment Used in diverse ways in different countries Useful to focus on incentive effects of payments, not just on predicting the right average payment Increasing importance in today s pay-for-performance markets Lots of interesting questions remain Page 23 Randall P. Ellis March 7, 2008
Results from three other projects Page 24 Randall P. Ellis March 7, 2008
Ellis and McGuire JHE (2007) Key Analytical Result The incentive to ration a service is determined by three things: predictability (measured by the coefficient of variation of expected service spending) predictiveness (measured by the correlation of expected service spending with total spending) Elasticity of demand for service s (η s ) Services with a large selection index will tend to be rationed more tightly than average while others will be loosely rationed. Page 25 Randall P. Ellis March 7, 2008
3.0 Figure 2A-1 Plot of selection indices versus relative spending by plan type / type of service Plan spending on each service relative to mean spending on service 2.5 2.0 1.5 1.0 0.5 COMP PPO POS HMO Linear (COMP) Linear (PPO) Linear (POS) Linear (HMO) 0.0 0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 Selection indices, all plan types Jiang, Ellis, and Kuo 2007 Page 26 Randall P. Ellis March 7, 2008
2.0 Figure 2A-2 Plot of selection indices versus relative spending by plan type / type of service, omitting all services with selection scores greater than 1.5 Plan spending on each service relative to mean spending on service 1.8 1.6 1.4 1.2 1.0 0.8 0.6 0.4 0.2 COMP PPO POS HMO Linear (COMP) Linear (PPO) Linear (POS) Linear (HMO) 0.0 0.0 0.2 0.4 0.6 0.8 1.0 1.2 Selection indices, all plan types Jiang, Ellis, and Kuo 2007 Page 27 Randall P. Ellis March 7, 2008
DxCG update: DxCG is updating its DCG/HCC classification system Current Release 6 New Release 7 2002 2008-2009 ICD-9-CM ICD-9-CM and ICD-10 3 million people 14 million people 184 HCCs ~ 400 HCCs 435 DxGroups ~ 800 DxGroups 30 hierarchies ~ 50 hierarchies Chronic/non-chronic conditions DxCG, Inc. 2008 Page 28 Randall P. Ellis March 7, 2008
Psychiatric Disorders V.6 Old DxCG, Inc. 2008 Page 29 Randall P. Ellis March 7, 2008
Psychiatric Disorders V.7 (Concurrent) New Solid green boxes Non-chronic Dashed red boxes Chronic DxCG, Inc. 2008 Page 30 Randall P. Ellis March 7, 2008
Heart V.6 Old DxCG, Inc. 2008 Page 31 Randall P. Ellis March 7, 2008
Heart V.7 (Concurrent) Artificial Heart / Assist Device Status Heart Transplant Complications New Other Diseases of Pulmonary Circulation Acute Heart Failure and CHF Exacerbation Acute Myocardial Infarction Heart Transplant Status Congestive Heart Failure Hypertensive Heart and Renal Disease or Encephalopathy Unstable Angina and Other Acute Ischemic Heart Disease Specified Heart Arrhythmias Major Congenital Cardiac/ Circulatory Defect Adhesive/ Constrictive Pericarditis and Cardiovascular Syphilis Hypertensive Heart Disease Heart Infection/ Inflammation, Except Rheumatic and Syphilitic Hypertension Angina Pectoris/ Old Myocardial Infarction Heart Valve Replacement / Transplant Coronary Atherosclerosis/ Other Chronic Ischemic Heart Disease Other Heart Rhythm and Conduction Disorders Congenital Heart and Circulatory Anomalies Chronic Valvular and Rheumatic Heart Disease Acute Rheumatic Heart Disease and Chorea Other and Unspecified Heart Disease DxCG, Inc. 2008 Page 32 Randall P. Ellis March 7, 2008
Diabetes V.7 New Type I Diabetes Mellitus Diabetes with Neurologic or Peripheral Circulatory Manifestation Diabetes with Renal Manifestation Type I Diabetes Mellitus Diabetes with Renal Manifestation Diabetes with Neurologic or Peripheral Circulatory Manifestation Concurrent Diabetes with Ophthalmologic Manifestation Diabetes with Ophthalmologic Manifestation Prospective Diabetes with Acute Complications Diabetes with Acute Complications Diabetes with No or Unspecified Complications Diabetes with No or Unspecified Complications DxCG, Inc. 2008 Page 33 Randall P. Ellis March 7, 2008
Work in progress Ellis and Mookim (2008) A within-sample method of validating predictive power with special application to risk adjustment models Traditional approach is to use split sample methods to evaluate overfitting. Inefficient in that only one split is traditionally considered. Predictive power is understated in validated measures of goodness of fit, for the same reason that fitted measures overstate power. Ellis and Mookim use systematic within-sample fitting and validation to generate more powerful measures. Page 34 Randall P. Ellis March 7, 2008
Prospective HCC model, fitted and validated R 2, by sample sizes Based on 100+ Monte Carlo draws of each size from 4.7 million US lives 40.0% 35.0% 30.0% 29.0% Mean fitted R2 5% fitted 95% fitted Mean validated R2 5% validated 95% validated R 2 (percent) 25.0% 20.0% 15.0% 23.8% 20.3% 16.2% 19.0% 18.3% 17.7% 17.6% 10.0% 5.0% 0.0% 10,000 20,000 50,000 100,000 200,000 500,000 1,000,000 Sample size Ellis and Mookim, 2008 Page 35 Randall P. Ellis March 7, 2008
Sources Ellis, R.P. (2008) "Risk adjustment in health care markets: concepts and applications" in Lu, Mingshan, and Jonnson, Egon, Paying for Health Care: New Ideas for a Changing Society. Wiley-VCH publishers Weinheim, Germany. Ellis, R.P. and McGuire, T.G. (2007) "Predictability and predictiveness in health care spending" Journal of Health Economics. 26:25 48. Jiang, S, Ellis, R.P., and Kuo, T-C. (2007) Does service-level spending show evidence of selection across health plan types? BU working paper. Ellis, R.P. and Mookim P.G. (2008) A within-sample method of validating predictive power with special application to risk adjustment models. (Working paper) Page 36 Randall P. Ellis March 7, 2008