ADVANCED STATISTICAL METHODS: PART 1: INTRODUCTION TO PROPENSITY SCORES IN STATA. Learning objectives:

Transcription

1 ADVANCED STATISTICAL METHODS: ACS Outcomes Research Course PART 1: INTRODUCTION TO PROPENSITY SCORES IN STATA Learning objectives: To understand the use of propensity scores as a means for controlling for selection bias in observational studies of treatment effects. To learn how to create propensity scores apply them in a variety of analytic approaches in STATA. To use propensity scores to evaluate the outcomes of open versus laparoscopic appendectomy in the NSQIP data provided. PROPENSITY SCORES IN STATA: Open the dataset and describe the data For this analysis, we will use NSQIP data for patients undergoing appendectomy ( ). This dataset contains 21,475 observations (patients) and 86 variables. The first group (2-56) includes preoperative patient characteristics and medical conditions. The second group (57-83) includes perioperative outcomes. The variables morbprob (var #84) and mortprob (var #85) are probabilities of morbidity and mortality, respectively, based on the NSQIP risk models. Finally, the variable treatment (var #86) tells you whether the appendectomy was performed open or laparoscopically. Type the command: describe, number Contains data from D:\nbirkmey\Desktop\Desktop\ACS Course 2010\Mark Propensity\Appendicitis_Propensity\ACS.NSQIP.Appy.dta obs: 21,475 vars: Oct :11 size: 4,123,200 (99.6% of memory free) variable storage display value name type format label variable label caseid long %12.0g Patient ID 2. sex byte %10.0g sex Patient Sex 3. race str32 %32s Patient Race 4. race1 byte %8.0g race==american Indian or Alaska Native 5. race2 byte %8.0g race==asian or Pacific Islander 6. race3 byte %8.0g race==black, Not of Hispanic Origin 7. race4 byte %8.0g race==hispanic, Black 8. race5 byte %8.0g race==hispanic, Color Unknown 9. race6 byte %8.0g race==hispanic, White 10. race7 byte %8.0g race==unknown 11. race8 byte %8.0g race==white, Not of Hispanic Origin 12. white byte %9.0g whitelab White/Non-White Race 13. white1 byte %8.0g white==white 14. white2 byte %8.0g white==non-white 15. white3 byte %8.0g white==unknown 16. prncptx str25 %25s Principal CPT 1

2 17. cpt str5 %5s CPT code 18. inout byte %10.0g inout Inpatient/Outpatient Status 19. age byte %8.0g Patient Age 20. operyr int %8.0g Year of Operation 21. height byte %8.0g Patient Height 22. weight int %8.0g Patient Weight 23. bmi float %9.0g BMI 24. bmi40 float %9.0g Morbid Obesity 25. bmicat byte %8.0g 4 quantiles of bmi 26. diabetes byte %10.0g diabetes DM/oral or insulin 27. diabet~1 byte %8.0g diabetes==no 28. diabet~2 byte %8.0g diabetes==oral Med 29. diabet~3 byte %8.0g diabetes==insulin 30. smoke byte %10.0g yesno Current (Within 1 YR) Smoker 31. etoh byte %10.0g yesno Current (Within 2 Weeks) Drinker (>2 per Day) 32. dyspnea byte %17.0g dyspnea Dyspnea 33. dyspnea1 byte %8.0g dyspnea==no 34. dyspnea2 byte %8.0g dyspnea==moderate Exertion 35. dyspnea3 byte %8.0g dyspnea==at Rest 36. dnr byte %10.0g yesno DNR Status 37. fnsta~s2 byte %19.0g fnstatus2 Functional Status Prior to Surgery 38. fnsta~21 byte %8.0g fnstatus2==independent 39. fnsta~22 byte %8.0g fnstatus2==partially Dependent 40. fnsta~23 byte %8.0g fnstatus2==totally Dependent 41. hxcopd byte %10.0g yesno History of Severe COPD 42. ascites byte %10.0g yesno Ascites 43. hxchf byte %10.0g yesno CHF within 30 days 44. hxmi byte %10.0g yesno MI within 6 mths 45. hypermed byte %10.0g yesno HTN requiring medication 46. dialysis byte %10.0g yesno On dialysis pre-op 47. pregna~y byte %10.0g yesno Pregnant at time of surgery 48. rupture byte %9.0g Rupture 49. emergncy byte %10.0g yesno Emergency Case 50. asa byte %18.0g asa ASA Class 51. asa1 byte %8.0g asaclas==1-no Disturb 52. asa2 byte %8.0g asaclas==2-mild Disturb 53. asa3 byte %8.0g asaclas==3-severe Disturb 54. asa4 byte %8.0g asaclas==4-life Threat 55. asa5 byte %8.0g asaclas==5-moribund 56. asa6 byte %8.0g asaclas==null 57. tothlos int %8.0g LOS 58. supinfec byte %10.0g yesno Superficial surgical site infection 59. wndinfd byte %10.0g yesno Deep Incisional SSI 60. orgspc~i byte %10.0g yesno Organ Space SSI 61. dehis byte %10.0g yesno Wound Disruption 62. oupneumo byte %10.0g yesno Pneumonia 63. reintub byte %10.0g yesno Unplanned Reintubation 64. pulembol byte %10.0g yesno PE 65. failwean byte %10.0g yesno Ventilator>48 Hours 66. renainsf byte %10.0g yesno Progressive Renal Insufficiency 67. oprenafl byte %10.0g yesno Acute Renal Failure 68. urninfec byte %10.0g yesno UTI 69. cnscva byte %10.0g yesno Stroke/CVA 70. cnscoma byte %10.0g yesno Coma>24 Hours 71. neurodef byte %10.0g yesno Peripheral Nerve Injury 72. cdarrest byte %10.0g yesno Cardiac Arrest req CPR 73. cdmi byte %10.0g yesno MI 74. othbleed byte %10.0g yesno Bleeding req. >4 units 75. othdvt byte %10.0g yesno DVT 76. othsysep byte %10.0g yesno Sepsis 77. othses~k byte %10.0g yesno Septic Shock 78. convert byte %9.0g yesno Convert Lap to Open 79. return byte %9.0g yesno Return to OR 80. died byte %9.0g diedlab Peri-op Death 81. anycomp float %9.0g Any Complication 82. mincomp float %9.0g Minor Complication 2

3 83. majcomp float %9.0g Major Complication 84. morbprob float %9.0g Morbidity Risk 85. mortprob float %9.0g Mortality Risk 86. treatment byte %9.0g treatment Open/Lap Treatment Sorted by: r Exploring the relationship between patient pre-operative characteristics and conditions and treatment We will first explore the relationship between patient pre-operative characteristics and treatment with either open or laparoscopic appendectomy in this dataset. Begin by tabulating the treatment variable. Type the command: tab treatment. tab treatment Open/Lap Treatment Freq. Percent Cum Lap Appy 15, Open Appy 5, Total 21, This output shows that 5,818 (27%) of patients had open appendectomy and 15,657 (5,818) patients had laparoscopic appendectomy. To determine what variables are associated with treatment you could do crosstabs for each covariate. For example, here is the crosstab for treatment and sex: Type the command: tab treatment sex, row chi nokey. tab treatment sex, row chi nokey Open/Lap Patient Sex Treatment Female Male Total Lap Appy 7,541 8,116 15, Open Appy 2,468 3,350 5, Total 10,009 11,466 21, Pearson chi2(1) = Pr =

4 Alternatively, you could fit a full model (include all covariates) or fit a stepwise model to obtain a more parsimonious model. sw, pe(0.05) : logistic treatment sex white age operyr bmi40 diabetes2 diabetes3 smoke etoh dyspnea2 dyspnea3 fnstatus22 fnstatus23 hxcopd ascites hxchf hxmi hypermed dialysis pregnancy emergncy asa begin with empty model p = < adding rupture p = < adding operyr p = < adding white p = < adding age p = < adding sex p = < adding emergncy p = < adding pregnancy p = < adding etoh p = < adding fnstatus23 p = < adding hxmi p = < adding dialysis Logistic regression Number of obs = LR chi2(11) = Prob > chi2 = Log likelihood = Pseudo R2 = treatment Odds Ratio Std. Err. z P> z [95% Conf. Interval] rupture operyr white age sex emergncy pregnancy etoh fnstatus hxmi dialysis The results show that treatment is associated with rupture, operative year, race, age, sex, emergency surgery, pregnancy, alcohol use, functional status, history of MI, and dialysis. We will use these variables to estimate the propensity score. predict prop (option pr assumed; Pr(treatment)) 4

5 tab treatment, sum(prop). tab treatment, sum(prop) Open/Lap Summary of Pr(treatment) Treatment Mean Std. Dev. Freq Lap Appy Open Appy Total This shows that the propensity score is about 25% in the lap appendectomy and about 31% in the open appendectomy treatment group. Analytic approach 1: Multivariate Modeling One way to use a propensity score is to simply add it as a covariate to a multivariate model. Here we will do that with mortality as the outcome. First we will fit a logistic model with treatment as the only covariate. logistic died treatment Logistic regression Number of obs = LR chi2(1) = Prob > chi2 = Log likelihood = Pseudo R2 = died Odds Ratio Std. Err. z P> z [95% Conf. Interval] treatment We see that patients undergoing open appy are 3.1 times more likely to die than patients undergoing laparoscopic appy. Now we will add the propensity score to the model. logistic died treatment prop 5

6 Logistic regression Number of obs = LR chi2(2) = Prob > chi2 = Log likelihood = Pseudo R2 = died Odds Ratio Std. Err. z P> z [95% Conf. Interval] treatment prop The addition of the propensity score reduced the treatment odds ratio to 1.4 and it is no longer statistically significant. Finally, we will add other predictors of death to the model. logistic died treatment prop mortprob Logistic regression Number of obs = LR chi2(3) = Prob > chi2 = Log likelihood = Pseudo R2 = died Odds Ratio Std. Err. z P> z [95% Conf. Interval] treatment prop mortprob 1.27e e e+08 We see that the treatment odds ratio has been further reduced to Analytic approach 2: Matching Now we will match patients in each treatment on propensity score using the psmatch2 command in STATA. We will specify the logit option, otherwise it would use probit which is the default. We will specify the common support option so that it will leave out cases that lie outside the range of propensity scores of the controls. We will use a caliper matching algorithm with a caliper distance of.005 and specify the no replacement option so that controls are only used once in the matching. We will look at the difference among the treated and control patients between the matched and unmatched cohorts for the following outcomes: any complication, minor complication, major complication, mortality, and return to the OR. psmatch2 treatment, pscore(prop) logit common caliper(.005) noreplacement out(anycomp mincomp majcomp died return) 6

7 There are observations with identical propensity score values. The sort order of the data could affect your results. Make sure that the sort order is random before calling psmatch Variable Sample Treated Controls Difference S.E. T-stat anycomp Unmatched ATT mincomp Unmatched ATT majcomp Unmatched ATT died Unmatched ATT return Unmatched ATT Note: S.E. does not take into account that the propensity score is estimated. psmatch2: psmatch2: Common Treatment support assignment Off suppo On suppor Total Untreated 0 15,657 15,657 Treated 110 5,708 5, Total ,365 21,475 We see that differences in rates of any complication, minor complication, and major complication are reduced but are still statistically significant (T-stat > 2.5). There are no longer statistically significant differences in rates of mortality or return to OR. To get an odds ratio for mortality based on the matched cohort like this. logistic died treatment if _weight==1 Logistic regression Number of obs = LR chi2(1) = 1.21 Prob > chi2 = Log likelihood = Pseudo R2 = died Odds Ratio Std. Err. z P> z [95% Conf. Interval] treatment

8 The way that you tell stata to use the matched cohort is using the _weight variable which was automatically generated by the psmatch2 command. Analytic approach 3: Stratification pscore treatment rupture operyr white age sex emergncy pregnancy etoh fnstatus23 hxmi dialysis, pscore(myps) blockid(mybid) comsup logit **************************************************** Algorithm to estimate the propensity score **************************************************** The treatment is treatment Open/Lap Treatment Freq. Percent Cum Lap Appy 15, Open Appy 5, Total 21, Estimation of the propensity score Iteration 0: log likelihood = Iteration 1: log likelihood = Iteration 2: log likelihood = Iteration 3: log likelihood = Logistic regression Number of obs = LR chi2(11) = Prob > chi2 = Log likelihood = Pseudo R2 = treatment Coef. Std. Err. z P> z [95% Conf. Interval] rupture operyr white age sex emergncy pregnancy etoh fnstatus hxmi dialysis _cons

9 Note: the common support option has been selected The region of common support is [ , ] Description of the estimated propensity score in region of common support Estimated propensity score Percentiles Smallest 1% % % Obs % Sum of Wgt % Mean Largest Std. Dev % % Variance % Skewness % Kurtosis ****************************************************** Step 1: Identification of the optimal number of blocks Use option detail if you want more detailed output ****************************************************** The final number of blocks is 15 This number of blocks ensures that the mean propensity score is not different for treated and controls in each block ********************************************************** Step 2: Test of balancing property of the propensity score Use option detail if you want more detailed output ********************************************************** Variable white is not balanced in block 4 Variable operyr is not balanced in block 5 Variable operyr is not balanced in block 9 The balancing property is not satisfied Try a different specification of the propensity score ******************************************* End of the algorithm to estimate the pscore ******************************************* 9

10 We see that the patients were grouped into 15 blocks and that within those blocks there is no difference in the mean propensity score for treated and controls. However, we are warned that there are some differences with regard to individual variables in several blocks. To address this, we will adjust for these variables in the next step. To evaluate the within strata difference in rates of mortality between treated and control patients we will use a conditional fixed effects model. First we must specify the panel variable for the fixed effect model which is the strata variable (mybid). xtset mybid panel variable: mybid (unbalanced) xtlogit died treatment white operyr, fe or nolog note: multiple positive outcomes within treatments encountered. note: 1 treatment (2301 obs) dropped because of all positive or all negative outcomes. Conditional fixed-effects logistic regression Number of obs = Treatment variable: mybid Number of treatments = 13 Obs per treatment: min = 6 avg = max = 6030 LR chi2(3) = Log likelihood = Prob > chi2 = died OR Std. Err. z P> z [95% Conf. Interval] treatment white operyr Once again we see that we get a statistically non-significant OR of about 1.5 for mortality. 10