Time-dependent propensity score and collider-stratification bias: an example of beta 2 -agonist use and the risk of coronary heart disease

Transcription

1 Time-dependent propensity score and collider-stratification bias: an example of beta 2 -agonist use and the risk of coronary heart disease M. Sanni Ali, Rolf H. H. Groenwold, Wiebe R. Pestman, Svetlana V. Belitser, Arno W. Hoes, A. de Boer & Olaf H. Klungel European Journal of Epidemiology Affiliated to the European Epidemiology Federation ISSN Eur J Epidemiol DOI /s

2 Your article is protected by copyright and all rights are held exclusively by Springer Science +Business Media Dordrecht. This e-offprint is for personal use only and shall not be selfarchived in electronic repositories. If you wish to self-archive your work, please use the accepted author s version for posting to your own website or your institution s repository. You may further deposit the accepted author s version on a funder s repository at a funder s request, provided it is not made publicly available until 12 months after publication. 1 23

3 Eur J Epidemiol DOI /s METHODS Time-dependent propensity score and collider-stratification bias: an example of beta 2 -agonist use and the risk of coronary heart disease M. Sanni Ali Rolf H. H. Groenwold Wiebe R. Pestman Svetlana V. Belitser Arno W. Hoes A. de Boer Olaf H. Klungel Received: 17 September 2012 / Accepted: 9 January 2013 Ó Springer Science+Business Media Dordrecht 2013 Abstract Stratification and conditioning on time-varying cofounders which are also intermediates can induce colliderstratification bias and adjust-away the (indirect) effect of exposure. Similar bias could be expected when one conditions on time-dependent PS. We explored collider-stratification and confounding bias due to conditioning or stratifying on time-dependent PS using a clinical example on the effect of inhaled short- and long-acting beta 2 -agonist use (SABA and LABA, respectively) on coronary heart disease (CHD). In an electronic general practice database we selected a cohort of patients with an indication for SABA and/or LABA use and ascertained potential confounders and SABA/LABA use per three month intervals. Hazard ratios (HR) were estimated using PS stratification as well as covariate adjustment and compared with those of Marginal Structural Models (MSMs) in both SABA and LABA use separately. In MSMs, censoring was accounted for by including inverse probability of censoring weights.the crude HR of CHD was 0.90 [95 % CI: 0.63, 1.28] and 1.55 This study was conducted on behalf of PROTECT WP2 (Framework for pharmacoepidemiology studies, full list of collaborators in Appendix 2 ). The PROTECT consortium (Pharmacoepidemiological Research on Outcomes of Therapeutics by a European ConsorTium) is a private public partnership coordinated by the European Medicines Agency (EMA). M. Sanni Ali R. H. H. Groenwold W. R. Pestman S. V. Belitser A. de Boer O. H. Klungel (&) Department of Pharmacoepidemiology and Clinical Pharmacology, Utrecht Institute for Pharmaceutical Sciences, University of Utrecht, Utrecht, The Netherlands O.H.klungel@uu.nl R. H. H. Groenwold W. R. Pestman A. W. Hoes O. H. Klungel Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht, The Netherlands [95 % CI: 1.06, 2.62] in SABA and LABA users respectively. When PS stratification, covariate adjustment using PS, and MSMs were used, the HRs were 1.09 [95 % CI: 0.74, 1.61], 1.07 [95 % CI: 0.72, 1.60], and 0.86 [95 % CI: 0.55, 1.34] for SABA, and 1.09 [95 % CI: 0.74, 1.62], 1.13 [95 % CI: 0.76, 1.67], 0.77 [95 % CI: 0.45, 1.33] for LABA, respectively. Results were similar for different PS methods, but higher than those of MSMs. When treatment and confounders vary during follow-up, conditioning or stratification on time-dependent PS could induce substantial colliderstratification or confounding bias; hence, other methods such as MSMs are recommended. Keywords Bias Collider-stratification Confounding Coronary heart disease Cox model Inhaled beta 2 -agonist Observational Time-dependent propensity score Time-varying treatment Abbreviations ATC Anatomical therapeutic chemical classification system CHD Coronary heart disease CI Confidence interval COPD Chronic obstructive pulmonary disease DAG Direct acyclic graph GPRD General practice research database HR Hazard ratio ICPC International classification of primary care system IPTW Inverse probability of treatment weights LABA Long-acting beta 2 -agonist MI Myocardial infarction MSM Marginal structural model PS Propensity score SABA Short-acting beta 2 -agonist SD Standard deviation

4 M. Sanni Ali et al. Introduction Propensity score (PS) methods [1] have become a commonly used approach in observational studies to control for confounding bias in estimation of causal treatment effects [2, 3]. These methods are used to balance patient characteristics between treatment groups in point-treatment studies, where confounders and treatment are constant over time. However, in follow-up studies, too simple (yes or no) treatment ascertainment may result in non-differential misclassification in the presence of non-compliance and/or treatment switching [4, 5]. In addition, treatment groups may quickly become less comparable over the course of the study resulting in biased estimates of treatment effects, unless comparisons can be balanced over follow-up [5]. With the increased availability of prospectively gathered and computerized information on medical diagnoses and medication use, treatment status can be assessed on a daily basis, rendering time-varying analysis of drug use possible [5]. A time-dependent Cox proportional hazards model is the common approach in estimating the effect of timevarying treatment [6]. However, this approach may be biased in the presence of time-varying confounders that are affected by prior treatment [6, 7], because adjustment for time-varying confounders that are affected by prior treatment may adjust away part of the treatment effect and induce collider-stratification bias (Fig. 1) [8 11]. The use of PS in such longitudinal analysis of exposure is limited to using them as inverse probability of treatment weights (IPTW), in marginal structural models (MSMs) [12]. Collider-stratification bias is a bias that is introduced by conditioning or stratifying on a collider, a variable that is a common effect of two or more variables in a causal pathway [10, 11]. The inclusion of collider(s) in a PS model and subsequent stratification or regression adjustment using this PS in such a time-varying analysis of treatment and covariates could lead to similar bias, although it is not specifically assessed in clinical research and its impact is uncertain [13]. For example, in the causal diagram presented in Fig. 1, we consider the PS to be a summary of time-dependent confounders. It can be speculated that time-varying propensity score at time t (PS t ), which includes beta 2 -agonist (LABA) and anticholinergics use that are proxies for severity of COPD, is predicted by previous SABA use (SABA t-1 ) and itself predicts subsequent SABA use (SABA t ) and the risk of coronary heart disease (CHD t ). Hence, PS t is a collider (summary of one or more colliders and confounders) on the causal path from previous SABA use (SABA t-1 ) to CHD t through PS t (SABA t-1? PS t? SABA t? CHD t ). By conditioning on the time-dependent PS, a spurious path will be opened from the exposure (SABA t ) to the outcome (CHD t ), via the unmeasured common causes (U) of confounders and outcome (SABA t? PS t? U?CHD t ), hence, inducing collider-stratification bias. It will also adjust away the (indirect) effect of previous SABA use (SABA t-1 ) via later confounders (PS t ), which are also intermediates, resulting in a biased estimate of the net effect of SABA use on CHD [9, 10]. The objective of this study was to illustrate colliderstratification bias associated with time-dependent PS methods in a cohort of patients using inhaled short- and long-acting beta 2 -agonist (SABA and LABA) and the risk of non-fatal coronary heart disease (CHD). Effect estimates from time-dependent PS methods were compared with those of Robins MSMs [6 8] to control for time-varying confounders. The example of inhaled beta 2 -agonist use and the risk of CHD was chosen, because of the contradicting results in the literature [14 21]. In the literature, explanations for these discrepancies included lack of statistical adjustment for confounders such as severity of COPD [19], among others. However, severity of COPD (its proxies such as LABA or anticholinergic use) could be a timedependent confounder of the causal association between SABA use and CHD, hence, conditioning on it could lead to collider-stratification bias. Methods Data, design and study population Fig. 1 Directed acyclic graphs (DAGs) representing possible causal association among summary of time-varying potential confounders (PS), inhaled beta 2 -agonist (SABA), CHD and potentially unmeasured factors (U). t, current time; t-1 previous time. Potential confounders at time t (PS t ) are independent prognostic factors for coronary heart disease (CHD t ), predictors of subsequent inhaled beta 2 -agonist use (SABA t ) and intermediates on the path SABA t-1? PS t? CHD t As an illustrative example, we used data from the Netherlands University Medical Centre Utrecht General Practitioner Research Network. This is a computerized medical database that includes cumulative information on approximately 60,000 patients. Medical diagnoses are registered according to the international classification of primary care (ICPC) system. Prescriptions are registered using the

5 Time-dependent propensity score and collider-stratification bias anatomical therapeutic chemical (ATC) classification system. We used information from the period A cohort study was conducted in adults with an indication for inhaled beta 2 -agonist use (patients with a diagnosis of incident bronchitis, asthma, or COPD: ICPC codes, R78/ R91, R96, and R95 respectively). To obtain detailed information on baseline characteristics, only patients experiencing incident bronchitis, asthma or COPD from 01 April 1995 onwards were included. Follow-up began the first day of diagnosis of bronchitis, asthma, or COPD and ended at the occurrence of either non-fatal CHD, death, loss to follow-up (unregistered with the GP), or end of the study (31 December 2005), whichever occurred first. Patients were excluded if they had any history of myocardial infarction (ICPC code K75) or angina pectoris (ICPC code K74) prior to or at the start of follow up. Outcome, exposure, and confounder assessment The primary outcome was defined as the first diagnosis of non-fatal myocardial infarction (MI; ICPC code, K75) or angina pectoris (AP; ICPC code, K74) and is referred to as coronary heart disease (CHD) in the rest of the manuscript. If both events were observed in the same patient, the earlier date of diagnosis was considered. Patients who died (possibly due to a fatal MI) were first excluded from the analysis because cause of death was not routinely registered [19] and later included as censored observations or combined with the end point (thus as events) in two separate sensitivity analyses to check whether the exclusion had any impact on the effect estimate. Using data on prescription dispensing date, we ascertained exposure status (inhaled beta 2 -agonist use) for every patient in terms of binary indicators in each three-month interval. A patient was considered a user (exposed) if he or she had filled at least one inhaled beta 2 -agonist prescription in the three-month interval (a nonuser or unexposed, otherwise). The choice for a three-month interval was based on the fact that Dutch health insurance policies cover the dispensing of the majority of drugs for three months [19]. In this study, we considered both inhaled short-acting beta-agonist (SABA) use [names (ATC codes): Salbutamol (R03AC02), Terbutaline (R03AC03), Fenoterol (R03AC04), Rimiterol (R03AC05), Fenoterol and other drugs for obstructive airway diseases (R03AK03) or Salbutamol and other drugs for obstructive airway diseases (R03AK04)] and long-acting beta-agonist (LABA) use [ATC: Salmeterol (R03AC12), Formeterol (R03AC13), Salmeterol and other drugs for obstructive airway diseases (R03AK06) or Salmeterol and other drugs for obstructive airway diseases (R03AK07)] as time-varying treatment in separate analyses. To evaluate this treatment classification, standard risk-set analysis was performed whereby a risk-set was constructed each time an event (CHD) occurred. At each of those time-points, treatment status as well as covariate values were ascertained for all patients at risk in the cohort. More details on the risk-set approach are included in Appendix 1. Information on potential confounders was available at baseline and during follow-up. The following potential confounders were available for analysis: age, gender, cardiovascular disease status (hypertension, heart failure, atrial fibrillation, paroxysmal tachycardia, cardiac arrhythmia, heart/atrial murmurs, pulmonary heart disease, heart valve diseases, other heart disease), presence of COPD, diabetes, inhalation glucocorticoids, anticholinergics, systemic corticosteroids, cardiovascular drugs (antithrombotic drugs, cardiac therapy, diuretics, agents acting on the renninangiotensin system), beta-blockers, statins, anti-diabetics, SABA and LABA use. In addition, cardiovascular drugs were pooled into a single binary variable indicating cardiovascular medication use. For chronic diseases such as COPD and diabetes, patients were classified as having the disease from the first date of diagnosis through follow up. Analysis We conducted three sets of analyses. In the first set of analyses, treatment was considered time-varying over the three-month intervals and all other covariates were considered constant from baseline onwards. In this case, the PS was estimated as the probability of inhaled SABA/LABA use in one or more three-month intervals in the follow-up period (i.e., ever vs. never use) conditional on observed baseline covariates. Hence, the PS was considered constant during follow-up. Then, treatment effects were estimated using the PS as a covariate and stratifying variable in a Cox model. In addition, the IPTW approach was used in which the estimated PS was used to assign weights to all observations (person-times). This weighting creates an altered composition of study population in which the probability of receiving LABA/SABA at each three-month interval is unrelated to confounders [6, 9]. The weight for each patient was the inverse of the probability that the patient had the treatment that he or she actually received. Hence, the weight for treated observations was 1/PS and for untreated observations 1/(1-PS). Finally, a marginal Cox model was fitted using inhaled SABA/LABA use as the only covariate on the altered study population. Marginal frequency of SABA/LABA use was used in the numerator of IPTW (instead of 1.0) to stabilize the weights [6 8]. In both PS and IPTW approaches, the outcome model included a time-varying binary variable for treatment, which indicated treatment status during a three-month interval. This approach adjusted for baseline confounders in the presence of time-varying treatment; hence, collider-stratification bias is not an issue here, but inadequate adjustment for confounding may invalidate the results.

6 M. Sanni Ali et al. Second, both treatment and covariates were considered time-varying at intervals of three months. In this approach, the data was restructured in a way that the longitudinal information of each patient was split-up into personmoments of three-month intervals, which included start and end dates, exposure status during that period, as well as covariates and censoring or event status (indicator values, i.e. yes = 1orno= 0, were used for covariate, treatment, censoring and event status in each three-month interval). Both exposure and covariates were assumed to be constant during the intervals, and covariate histories were actualized so that their values are temporally prior to treatment. Crude and adjusted risks of CHD associated with inhaled SABA or LABA use was estimated using (multivariable) Cox proportional hazards model. In addition, the propensity for inhaled SABA/LABA use was estimated for each threemonth interval by fitting logistic regression model that included the thirteen demographic and clinical variables listed in Table 1. The PS was defined as the probability of exposure to inhaled SABA/LABA during a three-month interval, conditional on observed covariates and exposure in the previous three-month interval. Hence, for each patient the PS can differ between consecutive three-month intervals. Then, treatment effect were estimated by fitting a Cox model that either included the PS as a continuous covariate, or stratified person-times on quintiles or deciles of the PS. Separate analyses were conducted for SABA and LABA use, and we only considered a simple PS model without any interaction or higher order terms. These methods could lead to biased estimates in the presence of time-dependent confounders that are affected by prior treatment. As a sensitivity analysis to assess the impact of using three-month intervals approach for classification of treatment status and potential confounders on the effect estimate, a risk-set analysis was performed. More details on the risk-set analysis can be found in Appendix 1. Third, similar to the previous set of analyses, both treatment and covariates were considered time-varying at three-month intervals, but marginal structural models were used. Two MSMs (one with only treatment weights and the second with combined treatment and censoring weights) were fitted. Stabilized treatment weights (Sw i ) and censoring weights (Cw i ) were calculated using the method described by Hernan et al. [7]. Inverse probability of treatment weight at each time t was defined as, Sw i ðtþ ¼ Yt k¼0 PrðAðkÞ¼a i ðkþaðk 1Þ¼a i ðk 1ÞÞ PrðAðkÞ¼a i ðkþaðk 1Þ¼a i ðk 1Þ;LðkÞ¼l i ðkþþ ð1þ where the numerator and denominator represent the probability of SABA/LABA use (A(k)) for each patient i at each three-month interval k (A(k) = a i (k)) given previous SABA or LABA use, Aðk 1Þ without and also with conditioning on time-varying covariates (LðkÞ), respectively. Inverse probability of censoring weights were estimated in the same way, except that the numerator and denominator represent the probability of remaining uncensored (C(k)) up to time t given past SABA/ LABA use, Aðk 1Þ without and with also conditioning on time-varying covariates, LðkÞ respectively: Cw i ðtþ ¼ Yt k¼0 PrðCðkÞ ¼0Cðk 1Þ ¼0; Aðk 1Þ ¼a i ðk 1ÞÞ PrðCðkÞ ¼0Cðk 1Þ ¼0; Aðk 1Þ ¼a i ðk 1Þ; LðkÞ l i ðkþ ð2þ Separate logistic regression models were fitted for the numerator and denominator. Treatment and censoring weights were then multiplied to get overall weights (Sw i ) in each three-month interval, Sw i (t) = Sw i (t) 9 Sw i (t). Informally, the denominator of Sw i (t) is the probability that a subject had the observed history of SABA/LABA and censoring up to time interval t. All analyses were performed in R, version [22] and correlation between observations was taken into account in both PS and Cox analyses using the cluster function. MSMs theoretically do not suffer from collider-stratification bias, since the confounding effect of time-dependent confounders that are affected by prior treatment is controlled by weighting instead of conditioning. In all analytic methods, we assumed exchangeability (i.e. no unobserved confounding or non-informative censoring), consistency (an individual s potential outcome under his or her observed treatment history is precisely his or her observed outcome), positivity (i.e. at every level of the confounders, individuals in the population have a nonzero probability of receiving every level of treatment, which implies that the average causal effect of the treatment can be estimated in each subset of the population defined by the confounders), and correct model specification. For further details on these assumptions, we refer to the literature [1, 6, 7, 23]. Results In total, 8,099 patients met the inclusion criteria specified in the Methods section and data on these subjects was used for analysis. A total of 337 (4.2 %) patients experienced CHD during a mean follow up of 4.5 years. Males comprised 42.8 % of the cohort and the mean age at start of follow-up was 49.6 (SD = 19.1) years. At some point in time during follow-up, 31 % and 15.6 % of the patients used inhaled SABA and LABA, respectively. Baseline

7 Time-dependent propensity score and collider-stratification bias Table 1 Baseline characteristics of patients by beta 2 -agonist use through follow up Characteristics SABA LABA Ever users (N = 3160) Never users (N = 4349) Ever users (N = 1264) Never users (N = 6835) Mean (SD) age (years) 44.6 (17.8) 52.8 (19.2)* 51.9 (18.4) 49.2 (19.2)* Male gender 1297 (41.0) 2169 (43.9) 579 (45.8) 2887 (42.2) Co-morbidities COPD 662 (20.9) 556 (11.3)* 595 (47.1) 623 (9.1)* DM 241 (7.6) 711 (14.4) 114 (9.0) 597 (8.7) CVD 835 (26.4) 1729 (35.0) 474 (37.5) 2090 (30.60* Co-medications Anti-diabetics 191 (6.0) 410 (8.3)* 103 (8.1) 498 (7.3) CV medications a 951 (30.1) 1865 (37.8)* 573 (45.3) 2243 (932.8)* Beta-blockers 571 (18.1) 1170 (23.7)* 267 (21.1) 1474 (21.6) Statins 90 (2.8) 229 (4.6)* 38 (3.0) 281 (4.1) Corticosteroids 451 (14.3) 558 (11.3)* 249 (19.7) 760 (11.1)* Anticholinergics 709 (22.4) 945 (19.1)* 579 (45.8) 1075 (15.7)* Glucocorticoids 1916 (60.6) 1025 (20.8)* 835 (66.0) 2088 (30.5)* SABA 834 (66.0) 2326 (34.0)* LABA 834 (26.4) 430 (8.7)* CVD Cardiovascular diseases (hypertension, heart failure, atrial fibrillation, cardiac arrhythmia, paroxysmal tachycardia, heart/atrial murmurs, pulmonary heart disease, heart valve diseases, other heart disease) * P values\0.05, P values were calculated using t test for continuous variable (age) and Chi square test for categorical variables. All variables are expressed as number of patients (percentages) except for age a CV Medications (antithrombotic drugs, cardiac therapy, diuretics, and agents acting on the rennin-angiotensin system) characteristics of patients included in the analysis are summarized in Table 1. Table 2 shows results of different PS and multivariable Cox analyses when only treatment was considered timevarying in three-month intervals and adjustment was made for covariates only at baseline (time-fixed covariates). There was sufficient overlap the PS distribution between treated and untreated subjects, except in the lower and upper quintiles or deciles of the PS (data not shown). Results from PS methods and multivariable Cox models were comparable (Table 2). Table 3 shows the crude and adjusted hazard ratios (HRs) for CHD associated with the use of inhaled SABA and LABA, when treatment and confounders were defined in the three-month intervals and adjustment was made for time-dependent confounders. The crude HR in case of inhaled SABA use was closer to unity and not significant (HR: 0.90 [95 % CI: 0.63, 1.28]) but in case of inhaled LABA use, it was significant (HR: 1.55 [95 % CI: 1.06, 2.62]). Once age was included in the model, further adjustments for other covariates in SABA use did not materially alter the HR. Similar results were obtained when treatment classification was based on the risk-set approach (crude HR: 0.91 [95 % CI: 0.56, 1.38] vs [95 % CI: 0.63, 1.28] for inhaled SABA and 1.74 [95 % CI: 1.13, Table 2 Adjusted estimates of hazard ratio for CHD associated with use of inhaled SABA and LABA using different PS (at baseline) methods with the three-month interval approach Methods SABA use LABA use HR 95 % CI HR 95 % CI PS stratification Quintiles of PS a , , 1.93 Deciles of PS b , , 1.76 PS covariate adjustment c , , 1.70 IPTW d , , 2.35 Multivariable Cox model , , 1.72 PS was estimated as the probability of SABA/LABA use in one or more of the three-month intervals during follow-up given patient characteristics only at base line In all cases, covariates (PS) and weights were considered constant over time (time-fixed covariates) a Stratification based on quintiles of PS in the Cox model b Stratification based on deciles of PS in the Cox model c PS were included as covariate in the Cox model d PS were used to assign (stabilized) weights in marginal Cox model 2.66] vs.1.55 [95 % CI: 1.06, 2.26] for inhaled LABA). Additional results from risk-set approach are included in the Appendix 1.

8 M. Sanni Ali et al. Table 4 shows the results of different time-dependent PS based Cox analyses and MSMs using the three-month interval approach. There was good overlap in the PS distribution between treated and untreated patients (data not shown). The means of the stabilized weights for both treatment and censoring were centered close to one for both SABA and LABA use. The stabilized treatment weights ranged from 0.02 to 8.79 for SABA use and from 0.12 to 3.79 for LABA use, respectively. Similarly, the stabilized weight for censoring ranged from 0.29 to The adjusted HR for CHD was 1.07 [95 % CI: 0.72, 1.60] and 1.13 [95 % CI: 0.76, 1.67] on quintile stratification of the PS in inhaled SABA and LABA users, respectively. There was no difference in the estimated effect when deciles of the PS were used instead of the quintile PS. Effect estimates from covariate adjustment using the PS were similar compared to quintile stratification based on the PS (HR: 1.09 [95 % CI: 0.74, 1.61] vs [95 % CI: 0.72, 1.60] for inhaled SABA, and 1.09 [95 % CI: 0.74, 1.62] vs [95 % CI: 0.76, 1.67] for inhaled LABA). Estimates from MSMs using combined treatment and censoring weights were lower compared to those using only treatment weights (HR 0.86 [0.55, 1.34] vs [0.60, 1.41] for inhaled SABA and 0.77 [0.45, 1.33]) vs [0.53, 1.50] for inhaled LABA, respectively). Inclusion of patients who died (possibly due to a fatal MI) did not affect the result (data not shown). Discussion Our goal was to illustrate collider-stratification and confounding bias associated with the use of time-dependent PS methods for the analysis of time-varying treatment, in Table 4 Estimates of hazard ratio for CHD associated with use of inhaled SABA and LABA using different time-dependent PS methods and MSMs With three-month interval approach Methods SABA use LABA use HR 95 % CI HR 95 % CI PS stratification Quintiles of PS a , , 1.67 Deciles of PS b , , 1.57 PS covariate adjustment c , , 1.62 MSMs-model 1 d , , 1.50 MSMs-model 2 e , , 1.33 In all cases, covariates except gender (PS) and weights were considered time-varying a Stratification based on quintiles of PS in the Cox model b Stratification based on deciles of PS in the Cox model c PS were included as covariate in the Cox model d PS was estimated as the probability of SABA/LABA use in each of the three-month interval during follow-up given patient characteristics in the previous three-month interval e Only stabilized treatment weight were used to fit MSMs f Both stabilized treatment and censoring weights were used to fit MSMs observational data, in the presence of time-dependent confounders. In empirical data, effect estimates from timedependent PS methods and MSMs of the association between inhaled SABA/LABA and the risk of CHD were different, suggesting the impact of conditioning on timedependent confounders that are also affected by prior treatment. Substantial confounding of the association between SABA use and CHD could not be displayed in our data set, as shown by similar effect estimates from crude and Table 3 (Un)adjusted estimates of hazard ratio (HR) for CHD associated with use of inhaled SABA and LABA using three-month interval (exposure classification) approach Adjusted for SABA use LABA use HR 95 % CI HR 95 % CI None (crude) , , 2.26 Age , , 1.92 Age?Gender , , 1.82 Age?Gender?CVD , , 1.81 Age?Gender?CVD?DM , , 1.81 Age?Gender?CVD?DM?COPD , , 1.59 Fully adjusted model a , , 1.43 In all cases, covariates were considered time-varying HR hazard ratio, CI Confidence interval a Confounders included in the model: Age, Gender, CVD, DM, COPD, Inhalation glucocorticoids, Anticholinergics, Systemic corticosteroids, Cardiovascular medications, Beta-blockers, Statins, Anti-diabetics, previous SABA/LABA use, and LABA in case of SABA use/saba in case of LABA use

9 Time-dependent propensity score and collider-stratification bias multivariable Cox models, time-varying PS methods and MSMs. However, there are important differences to note on LABA use and the risk of CHD. Estimates from PS models are higher than those of MSMs and in opposite direction although the confidence intervals (CIs) were overlapping. These differences could in part be explained by the fact that the time-dependent PS t (a function of time-varying anticholinergics or time-varying beta 2 -agonist use other than the treatment of interest, i.e. severity of COPD) is a collider of prior treatment and possible unobserved risk factor (U) for the CHD. Conditioning or stratification on this PS, like the time-dependent Cox model, may induce collider-stratification bias and also adjust away the (indirect) effect of previous treatment via time-dependent confounders (PS t ), which are also intermediates [9, 10]. Another possible explanation for the differences in treatment effect estimates is non-collapsibility [24, 25] of the HR. Conditional treatment effect estimate from Cox models that include the PS could be different from the marginal treatment effect from MSMs. However, the impact of non-collapsibility in our study is probably limited since the incidence of the outcome during follow up was relatively low (i.e., 4.2 %) [26]. In addition, PS methods give, in general, treatment effect estimates that are closer to the true marginal treatment effect than a conventional regression model in which all confounders are separately included in the adjusted model [26]. It could be argued that estimates obtained by conditioning or stratification on time-dependent confounders or PS t represent the direct effect of SABA or LABA use (SABA t-1 ) on CHD t and only adjust away its indirect effect through intermediates (PS t ). However, this does not hold true in the presence of unmeasured common causes of confounders (PS t ) and outcome (CHD t ) even in the absence of unmeasured confounding on SABA t use and CHD t, which is the underlying basic assumption in both PS methods and MSMs [1, 6, 7, 27, 28]. In our empirical study, whether the impact of confounding or collider-stratification on bias is largest could not be assessed. Bias induced by adjusting for a collider could be comparable or could result in estimates with opposite direction from the true effect thereby altering conclusions and not just the strength of an association [10, 11]. Both for SABA and LABA use, estimates from multivariable models were closer to MSMs than PS methods, which is in line with findings from simulation studies of point treatment settings [29 33]. Furthermore, we used only a simple PS model (all observed confounders included, without interactions or higher order terms) which may not result in the optimal balance of covariates between treatment groups. Thus, both confounding and colliderstratification may bias the observed effect estimates. Another possible source for bias is violation of the assumptions underlying our analyses, that the outcome model, the PS model for both treatment and censoring are correctly specified. Again, if loss to follow-up was related to treatment (beta 2 -agonist use) and outcome (CHD), results from timedependent PS methods would be more biased due to selective loss to follow-up. However, inclusion of the censoring weights in MSMs can help us back to one of the untestable assumptions of exchangeability in the standard Cox model that censoring is non-informative, again under the assumptions of no unmeasured confounding for treatment and censoring being only dependent on observed patient characteristics [6, 7]. In our study, censoring could be non-random since results from MSMs fitted with only treatment weights versus both treatment and censoring weights were different. However, it is difficult to make general conclusion in the context of a single study. In both cases, we used stabilized weights to normalize the range of these inverse probabilities and increase efficiency of the analysis [6, 33, 34]. The convergence of the mean of stabilized weight to unity and the overlap (common support) of the PS of the two treatment groups are an indirect support that the positivity assumption holds in our example. We did not consider weight truncation to reduce the effect of influential observations and variance of the treatment effect estimate since it could introduce residual confounding [23, 35]. Our study has both strengths and limitations. We think that the three-month interval approach for treatment ascertainment in this study minimizes treatment misclassification. A similar treatment ascertainment approach was also used in a US case control study that reported increased risk of unstable angina or myocardial infarction associated with beta 2 -agonist use [16] and another Dutch case control study that indicated no increased risk to users [19]. Moreover, results were similar when compared with the risk-set approach which does not restrict the time-axis to discrete three-month intervals. Nonetheless; residual treatment misclassification seems likely, since we used computerized records of prescriptions which may not reflect actual patient adherence. In addition, misclassification of clinical end points might be possible since diagnosis of CHD (MI) is usually made at hospitals and we used only GP data. Although our study has addressed several potential confounders in a time-varying pattern, we did not have complete information on important potential confounders such as body mass index, smoking, and severity of comorbidities, which may still bias the estimated effect. Notice that these potential confounders are very likely to change over time (i.e., could be time-varying) and they may be affected by prior treatment (e.g., severity of COPD may be affected by LABA use). Hence, residual

10 M. Sanni Ali et al. confounding due to unmeasured (time-dependent) confounders may still remain. However, the aim of this study was only to illustrate the use of time-dependent PS methods and potential consequences and not to answer the clinical research question: the effects of beta-agonist use on the risk of CHD. Therefore, the results should be interpreted with caution. In conclusion, in the presence of time-varying confounders that are affected by prior treatment, the use of time-dependent PS stratification or covariate adjustment, like the conventional time-dependent Cox model, can induce bias by collider-stratification adjusting-away the effect of treatment through intermediates. In such settings, other methods such as MSMs are more appropriate. Acknowledgments The research leading to these results was conducted as part of the PROTECT consortium (Pharmacoepidemiological Research on Outcomes of Therapeutics by a European ConsorTium, which is a public private partnership coordinated by the European Medicines Agency. The PRO- TECT project has received support from the Innovative Medicine Initiative Joint Undertaking ( under Grant Agreement n , resources of which are composed of financial contribution from the European Union s Seventh Framework Programme (FP7/ ) and EFPIA companies in kind contribution. The views expressed are those of the authors only. Conflict of interest The department of Pharmacoepidemiology and Clinical Pharmacology, Utrecht Institute for Pharmaceutical Sciences, has received unrestricted research funding from the Netherlands Organisation for Health Research and Development (ZonMW), the Dutch Health Insurance Board (CVZ), the Royal Dutch Association for the Advancement of Pharmacy (KNMP), the private public funded Top Institute Pharma ( includes co-funding from universities, government, and industry), the EU Innovative Medicines Initiative (IMI), EU 7th Framework Program (FP7), the Dutch Medicines Evaluation Board, the Dutch Ministry of Health and industry (including GlaxoSmithKline, Pfizer, and others). Appendix 1 To evaluate the sensitivity of the treatment-effect estimate to the three-month intervals treatment classification, standard risk-set analysis was performed. A risk-set was constructed based on occurrence of the event (CHD), then every time an event occurs, changes in treatment (SABA/ LABA use) status as well as other potential confounders were ascertained for each patient at risk in contrast to the three-month intervals approach. A patient could contribute several person-times in the risk-set and SABA/LABA use was not constrained by three-month interval but only occurrence of an event. A time-varying Cox model was then fitted on strata of the risk set (strata = number of unique events times). Table 5 shows the crude and adjusted HRs for CHD associated with the use of inhaled SABA and Table 5 (Un)adjusted estimates of hazard ratio (HR) for CHD associated with use of inhaled SABA and LABA using risk-set (exposure classification) approach Adjusted for SABA use LABA use LABA, when treatment and confounders were defined in the risk-set approach and adjustment was made for timedependent confounders. Similar results were obtained when treatment classification was based on the three-month interval approach (Table 3 of the manuscript). Appendix 2 HR 95 % CI HR 95 % CI None (crude) , , 2.66 Age , , 2.26 Age?Gender , , 2.12 Age?Gender?CVD , , 2.12 Age?Gender?CVD?DM , , 2.12 Age?Gender?CVD?DM?COPD , , 1.83 Fully adjusted model a , , 1.57 Effect estimates from the fully adjusted model using the three-month interval approach were 1.03 [0.69, 1.55] and 0.94 [0.62, 1.43] for SABA use and LABA use, respectively In all cases, covariates was considered time-varying HR hazard ratio, CI confidence interval a Confounders included in the model: Age, Gender, CVD, DM, COPD, Inhalation glucocorticoids, Anticholinergics, Systemic corticosteroids, Cardiovascular medications, Beta-blockers, Statins, Anti-diabetics, previous SABA/LABA use, and LABA in case of SABA use/saba in case of LABA use Members of PROTECT WP2 (Framework for pharmacoepidemiology studies): Y. Alvarez, J. Slattery, X. Kurz (European Medicines Agency), M. Rottenkolber, J. Hasford, A. Sassenfeld (Ludwig-Maximilians-Universität-München), F. J. (de) Abajo Iglesias, M. Gil, C. Huerta, D. Montero (Agencia Espanola de Medicamentos y Productos Sanitarios), L.A. Garcia-Rodriguez, A. Ruigomez (Fundación Centro Español de Investigación Farmacoepidemiológica), P. Souverein, D. de Bakker, A. de Boer, R. Groenwold, S. Belitser, W. Pestman, K. Roes, A. Hoes, V. Abbing-Karahagopian, F. de Vries, T.P. van Staa, A.C.G. Egberts, H.G.M. Leufkens, L. van Dijk, O.H. Klungel, M. De Groot, R. van den Ham, M. Sanni Ali, E. Voogd, M. J. Uddin (Utrecht University, The Netherlands), A. M. Gallagher, D. Dedman, J. Campbell (The UK General Practice Research Database), P. Helboe, J. Lyngvig, AM Clemensen, TS Engraff, U. Hesse, J. Poulsen (Lægemiddelstyrelsen, Danish Medicines Agency), John Logie, Jeanne Pimenta (GlaxoSmithKline Research and Development LTD), L. Bensouda-Grimaldi, L. Abenhaim (L.A.

11 Time-dependent propensity score and collider-stratification bias Sante Epidemiologie Evaluation Recherche), R.F. Reynolds, N. Gatto, A. Bate (Pfizer), G.F. Downey, R. Brauer, M. Schoonen, A. Roddam (Amgen NV), O. Demol (Genzyme Europe), M. Miret (Merck KgaA), S. Johansson (AstraZeneca AB), P. Primatesta, R. Schlienger, J. Fortuny, E. Rivero (Novartis), G. Quartey, H. Petri, M. Schuerch, J. Robinson (F.Hoffmann-La Roche AG), J.R. Laporte, L. Ibañez, M. Sabaté, E. Ballarin, P. Ferrer (Fundació Institut Català de Farmacologia). References 1. Rosenbaum PR, Rubin DB. The central role of the propensity score in observational studies for causal effects. Biometrika. 1983;70: Shah BR, Laupacis A, Hux JE, Austin PC. Propensity score methods gave similar results to traditional regression modeling in observational studies: a systematic review. J Clin Epidemiol. 2005;58: Stürmer T, Joshi M, Glynn RJ, Avorn J, Rothman KJ, Schneeweiss S. A review of the application of propensity score methods yielded increasing use, advantages in specific settings, but not substantially different estimates compared with conventional multivariable methods. J Clin Epidemiol. 2006;59: Marcus SM, Siddique J, Ten Have TR, Gibbons RD, Stuart E, Normand SLT. Balancing treatment comparisons in longitudinal studies. Psychiatr Ann. 2008;38: Stricker BHC, Stijnen T. Analysis of individual drug use as a time-varying determinant of exposure in prospective populationbased cohort studies. Eur J Epidemiol. 2010;25: Robins JM, Hernán MÁ, Brumback B. Marginal structural models and causal inference in epidemiology. Epidemiology. 2000;11: Hernán MÁ, Brumback B, Robins JM. Marginal structural models to estimate the causal effect of zidovudine on the survival of HIV-positive men. Epidemiology. 2000;11: Robins JM. Marginal structural models. Section on Bayesian Statistical Science: Proceedings or the American Statistical Association; p Robins JM, Greenland S. Identifiability and exchangeability for direct and indirect effects. Epidemiology. 1992;3: Greenland S. Quantifying biases in causal models: classical confounding vs collider-stratification bias. Epidemiology. 2003;14: Whitcomb BW, Schisterman EF, Perkins NJ, Platt RW. Quantification of collider-stratification bias and the birth weight paradox. Paediatr Perinat Epidemiol. 2009;23: Segal JB, Griswold M, Achy-Brou A, et al. Using propensity scores subclassification to estimate effects of longitudinal treatments: an example using a new diabetes medication. Med Care. 2007;45:S Westreich D, Cole SR, Funk MJ, Brookhart MA, Stürmer T. The role of the c-statistic in variable selection for propensity score models. Pharmacoepidemiol Drug Saf. 2011;20: Spitzer WO, Suissa S, Ernst P, et al. The use of b-agonists and the risk of death and near death from asthma. N Engl J Med. 1992;326: Au DH, Curtis JR, Every NR, McDonell MB, Fihn SD. Association between inhaled b-agonists and the risk of unstable angina and myocardial infarction*. Chest. 2002;121: Au DH, Lemaitre RN, Randall Curtis J, Smith NL, Psaty BM. The risk of myocardial infarction associated with inhaled beta-adrenoceptor agonists. Am J Respir Crit Care Med. 2000;161: Suissa S, Assimes T, Ernst P. Inhaled short acting b agonist use in COPD and the risk of acute myocardial infarction. Thorax. 2003;58: Salpeter SR, Ormiston TM, Salpeter EE. Cardiovascular effects of b-agonists in patients with asthma and COPD*. Chest. 2004;125: De Vries F, Pouwels S, Bracke M, et al. Use of b2 agonists and risk of acute myocardial infarction in patients with hypertension. Br J Clin Pharmacol. 2008;65: Sears MR. Safety of long-acting b-agonists. Chest. 2009;136: Zhang B, de Vries F, Setakis E, van Staa TP. The pattern of risk of myocardial infarction in patients taking asthma medication: a study with the general practice research database. J Hypertens. 2009;27: R Development Core Team. R: a language and environment for statistical computing. R foundation for statistical computing. Vienna, Austria. 2011; ISBN , URL R-project.org/. 23. Cole SR, Hernán MA. Constructing inverse probability weights for marginal structural models. Am J Epidemiol. 2008;168: Greenland S, Robins JM. Identifiability, exchangeability, and epidemiological confounding. Int J Epidemiol. 1986;15: Greenland S, Robins JM, Pearl J. Confounding and collapsibility in causal inference. Statist Sci. 1999;14: Martens EP, Pestman WR, de Boer A. Systematic differences in treatment effect estimates between propensity score methods and logistic regression. Int J Epidemiol. 2008;37(5): Rosenbaum PR, Rubin DB. Reducing bias in observational studies using subclassification on the propensity score. J Am Stat Assoc. 1984;79(387): Rubin DB. Estimating causal effects from large data sets using propensity scores. Ann Intern Med. 1997;127: Peduzzi P, Concato J, Feinstein AR, Holford TR. Importance of events per independent variable in proportional hazards regression analysis II. Accuracy and precision of regression estimates. J Clin Epidemiol. 1995;48: Peduzzi P, Concato J, Kemper E, Holford TR, Feinstein AR. A simulation study of the number of events per variable in logistic regression analysis. J Clin Epidemiol. 1996;49: Cepeda MS, Boston R, Farrar JT, Strom BL. Comparison of logistic regression versus propensity score when the number of events is low and there are multiple confounders. Am J Epidemiol. 2003;158: Vittinghoff E, McCulloch CE. Relaxing the rule of ten events per variable in logistic and Cox regression. Am J Epidemiol. 2007;165: Kurth T, Walker AM, Glynn RJ, et al. Results of multivariable logistic regression, propensity matching, propensity adjustment, and propensity-based weighting under conditions of nonuniform effect. Am J Epidemiol. 2006;163: Hernán MA, Hernández-Díaz S, Robins JM. A structural approach to selection bias. Epidemiology. 2004;15: Lee BK, Lessler J, Stuart EA. Weight trimming and propensity score weighting. PLoS ONE. 2011;6:e doi: /journal. pone