Performance of the disease risk score in a cohort study with policy-induced selection bias

Transcription

1 For reprint orders, please contact: Performance of the disease risk score in a cohort study with policy-induced selection bias Aim: To examine the performance of the disease risk score (DRS) in a cohort study with evidence of policy-induced selection bias. Methods: We examined two cohorts of new users of bisphosphonates. Estimates for 1-year hip fracture rates between agents using DRS, exposure propensity scores and traditional multivariable were compared. Results: The results for the cohort with no evidence of policy-induced selection bias showed little variation across analyses (-4.1 to 2.0%). Analysis of the cohort with evidence of policy-induced selection bias showed greater variation (-13.5 to 8.1%), with the greatest difference seen with DRS analyses. Conclusion: Our findings suggest that caution may be warranted when using DRS methods in cohort studies with policy-induced selection bias, further research is needed. Keywords: cohort studies confounding factors (epidemiology) data interpretation multivariate osteoporosis/epidemiology Background Administrative health databases are major sources of data for comparative effectiveness research; however, their use also presents unique challenges [1]. One such challenge is the possibility of selection bias introduced by differential reimbursement policies that limit access by payers [2 4]. Payers may decide to limit use or implement step-based coverage of newer agents resulting in a form of selection bias we describe as policy-induced selection bias. Indeed, as newer agents are introduced to the market for similar indications, they are often prescribed in sicker patients [3,5]. Analytical strategies to control for confounding related to selection bias have been an important area of development in medical research, specifically in pharmacoepidemiology [6 9]. Adjustment using confounder summary scores has gained popularity with the development of two methods: the exposure propensity score (EPS) and the disease risk score (DRS) [10 12]. Both scores are created by fitting multivariable regression models that include all potential measured confounders for the exposure (EPS) [8,13 15] or the outcome (DRS) [16,17]. Use of the EPS has been widespread, with a high level of utilization in the literature [10,13 14,18 19], whereas use of the DRS has been limited, but increasing [9]. Limited use is perhaps in part due to the less apparent benefits of the DRS, as well as concern due to early simulation work suggesting possible bias when using the DRS [9,10,16,20 21]. Simulation studies found that bias is potentially introduced with DRS use in situations where there was >90% correlation between a confounder and the exposure [20]. Various forms of selection bias will often systematically increase the level of correlation between confounders and the exposure and thus may inflate bias in the calculated estimates. Few empirical applications have used DRS in comparative effectiveness research, as the majority of DRS applications have been in studies with unexposed populations [9,22 24]. We examined the performance of the DRS in a comparative effectiveness study of oral bisphosphonates utilizing a cohort with known policy-induced selection bias. Mina Tadrous*,1,2,3, Muhammad M Mamdani 1,2,3,4, David N Juurlink 2,5, Murray D Krahn 1,2,4,6, Linda E Lévesque 2,7 & Suzanne M Cadarette 1,2 1 Leslie Dan Faculty of Pharmacy, University of Toronto, 144 College Street, Toronto, ON M5S 3M2, Canada 2 Institute for Clinical Evaluative Sciences, 2075 Bayview Avenue G1 06, Toronto, ON M4N 3M5, Canada 3 Applied Health Research Centre, Li Ka Shing Knowledge Institute, St Michael s Hospital, 30 Bond St, Toronto, ON M5B 1W8, Canada 4 Institute of Health Policy, Management & Evaluation, University of Toronto, 27 King s College Cir, Toronto, ON M5S, Canada 5 Sunnybrook Research Institute, 2075 Bayview Avenue, Toronto, ON M4N 3M5T, Canada 6 Toronto Health Economics & Technology Assessment (THETA) Collaborative, Toronto ON M5S 3M2, Canada 7 Department of Public Health Sciences, Queen s University, 99 University Ave, Kingston, ON K7L 3N6, Canada *Author for correspondence: Tel.: Fax: mina.tadrous@utoronto.ca part of /cer Future Medicine Ltd J. Comp. Eff. Res. (Epub ahead of print) ISSN

2 Tadrous, Mamdani, Juurlink, Krahn, Lévesque & Cadarette Methods We leveraged an established cohort of older community-dwelling residents in Ontario, Canada who were new users of oral bisphosphonates at doses approved for the treatment of osteoporosis [25]. Data utilized in this cohort were drawn from Ontario s comprehensive databases of healthcare utilization and pharmacy claims, and covers all of the province s more than 1.3 million residents aged 65 or older. These datasets were linked using unique, encoded identifiers and analyzed at the Institute for Clinical Evaluative Sciences [25]. The cohort comprised all new users of osteoporosis doses of oral bisphosphonates (alendronate, cyclical etidronate and risedronate) between 1 April 2002 and 31 March 2008, aged 66 or more years. We restricted our study to residents over the age of 66 years to ensure a minimum of a 1-year pharmacy claims history. A policy limiting the use of risedronate and alendronate to patients with more severe disease was in place until 2007, while etidronate was available without restriction (general access) throughout the entire study period. The index date (cohort entry) was defined as the first date an eligible oral bisphosphonate drug was dispensed. Cohort members were excluded if they had other disease of the bone (e.g., Paget s disease), malignancy or non-osteoporosis high dose bisphosphonate use. The outcome of interest was hip fracture within 12 months of treatment initiation. Hip fracture was selected due to its high level of validity using administrative data [26,27]. Trials have shown evidence of bisphosphonate efficacy on bone quality and fracture reduction within 12 months [18,28 29]. Baseline covariates included factors known to affect fracture risk, including age, sex, osteoporosis diagnosis, history of fractures, co-morbidities, drug history and hospitalization [25]. Age was determined at index date and pharmacy and medical services identified in the year prior to index. Prior fracture was identified within 5 years prior to index. Exposure propensity score & disease risk score derivation Two separate cohorts were created: all etidronate and alendronate users (cohort with possible evidence of policy-induced selection bias), all alendronate and risedronate (cohort with similar restrictions and no evidence of policy-induced selection bias). DRS and EPS scores were created separately in each cohort. DRS was created using a single Cox proportional hazard model predicting the outcome with all covariates included in the model. DRS was calculated within each cohort. Creating the DRS in the whole cohort, as opposed to one exposure group, has been shown to minimize bias. [10] DRS was calculated for each subject based on fitted model parameters using an estimate of the baseline hazard function at 12 months. DRS was then categorized into quintiles. EPS was created within each cohort using logistic regression predicting the probability of exposure (alendronate) with all covariates. EPS was calculated for each subject based on fitted model parameters. The EPS was then categorized into quintiles. Statistical All analyses were completed to compare hip fracture rates between new users of either etidronate or risedronate to alendronate and stratified by sex. The etidronate to alendronate comparison was used to examine the impact of policy-induced selection bias on the performance of the DRS, whereas the alendronate to risedronate comparison was used as the reference cohort since it demonstrated no evidence of policy-induced bias. Cox proportional hazard models were used to compare fracture rates between exposures in each cohort. Cohort members were censored at the earliest of end of follow-up (365 days after index) or date of death. Subjects were not censored on drug stoppage due to the persistence of bisphosphonate in bone beyond treatment termination [28,30]. A traditional multivariate model including all identified covariates was used as the base and compared with the DRS- and EPS-derived model estimates. DRS was modeled in the Cox proportional hazard regression and used in three distinct analyses: continuous variable, categorical (quintile) variable and stratified into quintiles. EPS was completed using stratification and matching since these are the most commonly used applications [18,31]. Matching with EPS was completed using the nearest neighbor based on greedy matching of the logit of the propensity score, using standard deviation caliper width of 0.2. All results were compared based on hazard ratio (HR) estimates. Percent deviation from the base was calculated to compare estimates. An additional sensitivity was conducted by trimming nonoverlap or 5% (2.5% on each tail) for both DRS and EPS applications. Results The cohort analyzed in this study consisted of 170,862 new users of bisphosphonates who experienced 2740 fracture events. The characteristics of this cohort have been previously described [12]. The event rate per 100 person-years among men was 1.23 for etidronate, 2.07 for alendronate and 1.98 for risedronate. The event rate per 100 person-years among women was 1.48 for etidronate, 1.94 for alendronate and 2.06 for risedronate /cer J. Comp. Eff. Res. (Epub ahead of print) future science group

3 Performance of the disease risk score in studies with selection bias Research Article Alendronate/risedronate cohort (no evidence of residual selection bias) The base using a traditional multivariable found no statistically significant difference in fracture risk between risedronate and alendronate in women (HR: ; 95% CI: ) (Table 1). In fact, little variation from the unadjusted model (3.9%) was found, with no significant difference between groups (HR: 1.06; 95% CI: ). Variation from the base was found to be least with DRS stratification (0.0%) and greatest when DRS was used as a continuous variable (2.0%). The most variance from the base was found when using DRS as a continuous variable, yet no statistically significant differences in fracture risk was identified (HR: 1.04; 95% CI: ). In men, the base found no statistically significant difference in fracture risk between risedronate and alendronate (HR: 0.98; 95% CI: ). Little variation from the unadjusted model (-2.0%) was found, with no significant difference between groups (HR: 1.00; 95% CI: ). Similar to the results observed for women, variation from the base was found to be least with DRS stratification (0.0%) and greatest when DRS was used as a continuous variable (-4.1%). The largest variance from the base was found when using DRS as a continuous variable, yet no significant difference was identified (HR: 0.94; 95% CI: ). Alendronate/etidronate cohort (evidence of possible residual selection bias) The base found no statistically significant difference in fracture risk between etidronate and alendronate in women (HR: 0.94; 95% CI: ), Table 1. Large variation from the unadjusted model (-18.1%) was found, with significant difference between groups in favor of etidronate (HR: 0.77; 95% CI: ). Variation from the base was found to be least with EPS matching (-1.1%) and greatest when the DRS was used as a continuous variable (-8.5%). All three DRS analyses showed a statistically significant finding in favor of etidronate. In contrast, both EPS analyses had nonsignificant findings. The base found statistically significant difference in fracture risk between etidronate and alendronate in men (HR: 0.74; 95% CI: ). Large variation from the unadjusted model (-16.2%) was found, with significant difference between groups in favor of etidronate (HR: 0.62; 95% CI: ). Variation from the base was least with EPS stratification (-1.4%) and greatest when the DRS was used as a continuous variable (-13.5%). All found statistically significant findings in favor of etidronate except for the EPS-matched, which found no difference (HR: 0.80; 95% CI: ). All three DRS analyses showed a statistically significant finding in favor of etidronate. ming was completed for the stratified DRS and was found to vary the results from 0.0 to -9.5% (Table 2). Discussion Our findings examine the use of the DRS in comparative effectiveness research and suggest possible implications of policy-induced selection bias. Results also corroborate prior cautions of using the DRS as a continuous variable [20,21] and continue the discussion of the impact of policy-induced selection bias in comparative effectiveness research [24]. DRS has been increasing in pharmacoepidemiologic research, particularly in comparative effectiveness studies [9] and studies of newly marketed agents [32]. In contrast, much of the initial development of the DRS concentrated on the choice of population for deriving the score, and the proper application of the DRS [10,11,16,24]. Previous findings have shown greater benefit with the use of the whole cohort for deriving the DRS, highlighted the limitations of using historical populations, [10,24,33 34] and shown that the DRS score should not be used as a continuous variable [20,21]. In addition, most of the DRS applications and development have studied unexposed comparison groups, [22] which are not easily extrapolated to comparative effectiveness research. The use of the DRS in comparative effectiveness research presents unique challenges related to market entry, as it has been shown that new agents are more likely to be utilized in sicker patients. This may in large part be due to reimbursement policy, prescribing comfort, depletion of susceptibles and possible compliance issues [35]. In our case example, limited-use criteria restricted access of newer bisphosphonates (alendronate and risedronate); to patients who had failed etidronate therapy (defined by continued bone loss), had contraindications to etidronate or exhibited decreased/severe bone health [36]. As a result of the limited use criteria, use of alendronate and risedronate was low until general access was granted in 2007 [36]. This limited use policy resulted in differential prescribing, with patients at higher risk of fractures being more likely to receive alendronate and risedronate. Evidence of use of newer bisphosphonates in sicker patients has been previously described both in terms of differential fracture rates, as well as differential baseline characteristics [25]. Our finding that the DRS may amplify differences in situations with policy-induced bias aligns with prior simulation work. For example, Pike et al. endorse caution when the exposure and confounders are highly future science group /cer.15.40

4 Tadrous, Mamdani, Juurlink, Krahn, Lévesque & Cadarette Table 1. Summary of results of and variation from base (multivariable ) and sex. Analysis Risedronate/alendronate (similar access cohort) Etidronate/alendronate (policy-induced selection bias cohort) Base traditional multivariable Hazard ratio ( ) Unadjusted 1.06 ( ) DRS continuous variable DRS categorical variable 1.04 ( ) ( ) DRS stratification ( ) EPS matched 1.03 ( ) EPS stratification 1.04 ( ) Women Men Women Men Change from base (%) Hazard ratio ) ( ) ( ) ) ) ( ) ( ) Change from base (%) Hazard ratio 0.94 ( ) ( ) ( ) ) ) ( ) ( ) Change from base (%) Hazard ratio 0.74 ( ) ( ) ( ) ( ) ( ) ( ) ( ) Change from base (%) This cohort represents a cohort where both drugs had similar access based on provincial payer policy and has no evidence of policy-induced selection bias. This cohort represents a cohort where both drugs have different access based on the provincial payers policy. This cohort exhibits possible signs of policy-induced selection bias. DRS: Disease risk score; EPS: Exposure propensity score /cer J. Comp. Eff. Res. (Epub ahead of print) future science group

5 Performance of the disease risk score in studies with selection bias Research Article correlated [10,20,37]. They showed that bias was introduced by the use of the DRS in simulations with 90% correlation between the confounder and exposure. In situations where selection bias is present, specific confounders are likely to be highly correlated with the exposure and will inflate due to residual confounding. Our results also affirm the need for caution when DRS is used as a continuous variable, which performed poorly in all analyses, as evidence by the larger variation in estimates from both conventional and EPS analyses. Early simulation work emphasized the need to apply the DRS as categorical variable to avoid introducing bias. This parallels similar findings with the EPS [18,20 21]. Even with previous recommendations, our previous systematic review found that the DRS was, in some cases, still applied as a continuous variable [9]. The major strength of our study is that we have a case example that allowed for two comparator cohorts within a similar period of time to contrast the effect of residual policy-induced selection bias. One cohort (etidronate/ alendronate) had evidence of residual confounding due to policy-induced bias while the other (risedronate/alendronate) did not. This allowed us the unique opportunity to contrast the performance of the DRS and EPS scores in both scenarios. In addition, this is the first study to our knowledge that addresses the issue of DRS performance in situations of policy-induced section bias. Our study is limited by the inability to capture information on unmeasured confounders such as over-thecounter use of calcium and vitamin D supplements and information on bone mineral density. Inclusion of all confounders will likely reduce the impact of selection bias on applications DRS analyses, and thus, impact our results. However, residual confounding is relevant in all studies utilizing administrative data and therefore highlights the importance of our findings. An additional limitation is that we do not know the true estimate of the risk difference between the bisphosphonates evaluated when used under real-world conditions. However, a number of head-to-head studies of varying designs have addressed this question and found no strong evidence to suggest differences in the effectiveness of bisphosphonates [25,38 40]. Future work to improve the use of the DRS in circumstances where selection bias may be common, such as in comparative effective research of new drugs, is needed. The potential benefits of the DRS in such studies are important, given the need to further improve the research of newly marketed agents. Our findings also raise question of use of the DRS in situations where confounding by indication is prevalent. Development of strategies for DRS application in situations of residual policy-induced selection bias would benefit a growing area of work. Our work also cautions against the often ignored impact of Table 2. Summary of results of sensitivity for disease risk score stratification reported as variation from base for trimming. Risedronate/alendronate (Similar access cohort) Etidronate/alendronate (Policy-induced selection bias cohort) Women Men Women Men DRS stratification Sensitivity DRS stratification Base DRS stratification Base DRS stratification Base Base 5% No trim Nonoverlap 5% No trim Nonoverlap 5% No trim Nonoverlap 5% No trim Nonoverlap 0.71 ( ) 0.67 ( ) 0.68 ( ) 0.74 ( ) 0.88 ( ) ) ) 0.94 ( ) 1.04 ( ) ) ) ) 1.07 ( ) ( ) ( ) ( ) Hazard ratios Change (%) This cohort represents a cohort where both drugs had similar access based on provincial payer policy and has no evidence of policy-induced selection bias. This cohort represents a cohort where both drugs have different access based on the provincial payers policy. This cohort exhibits possible signs of policy-induced selection bias. DRS: Disease risk score. future science group /cer.15.40

6 Tadrous, Mamdani, Juurlink, Krahn, Lévesque & Cadarette policy on comparative effectiveness research. Further methodological developments related to matching using the DRS, use of historical estimations, time-varying DRS and combinations of strategies are essential to the improvement of DRS applications [33,41 43]. Conclusion Our study suggests that DRS methods may exaggerate differences between newer and older drugs in the context of residual selection bias, favoring older drugs which are often used in healthier patients. This exaggeration may be due to a high level of correlation between exposure and confounders caused by selection bias. We also corroborate the recommendation to avoid use of the DRS as a continuous variable. Our findings caution researchers to be mindful of possible selection bias when conducting comparative effectiveness research. Future simulation work should explore ways to better use the DRS in the context of policy-induced selection bias. Future perspective Research, similar to this work, will continue to fine-tune and improve the use of confounder summary scores, such as the DRS. The improved use and better understanding of their strengths and weaknesses will be essential to producing high-quality comparative effectiveness research. Disclosure This work was presented at the Canadian Association for Population Therapeutics meeting, Toronto, November 2014; and the International Conference on Pharmacoepidemiology and Therapeutic Risk Management, Taipei, October Financial & competing interests disclosure This research was supported by a research grant to SM Cadarette from the Ontario Ministry of Research and Innovation Early Researcher Award (ER ). This work is part of M Tadrous s PhD dissertation which is currently under a 2-year embargo until publication. SM Cadarette was supported by a Canadian Institutes of Health Research (CIHR) New Investigator Award (MSH-95364), and M Tadrous was supported by a CIHR Fredrick Banting and Charles Best Canada Graduate Scholarship Doctoral Award (GSD-11342). MM Mamdani has served as an advisory board member for the following pharmaceutical companies: Astra Zeneca, Bristol-Myers Squibb, Eli Lilly and Company, GlaxoSmith Kline, Hoffman La Roche, Novartis, Novo Nordisk and Pfizer. The authors have no other relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript apart from those disclosed. No writing assistance was utilized in the production of this manuscript. Executive summary Disease risk score advantages Disease risk scores (DRS) are confounder summary scores derived on the probability of the outcome and may be advantageous over other adjustment techniques when studying newly marketed agents. Reimbursement policy as a form of selection bias Newer agents are more likely to be utilized in sicker patients; this may often be due to reimbursement policy. Policies such as limited use or step-based coverage of newer agents may result in a form of selection bias we term policy-induced selection bias. Performance of the disease risk score DRS may amplify differences in situations with policy-induced selection bias. This may be due to a high correlation between confounders and the exposure and thus may inflate already contained bias. This work supports the recommendation to avoid use of the DRS as a continuous variable. Our findings caution researchers to be mindful of possible policy-induced selection bias when conducting comparative effectiveness research. Future work should explore ways to better use the DRS in the context of policy-induced selection bias. References Papers of special note have been highlighted as: of interest; of considerable interest 1 Strom BL. Methodologic challenges to studying patient safety and comparative effectiveness. Med. Care 45(10 Suppl. 2), S13 S15 (2007). 2 Haneuse S. Distinguishing selection bias and confounding bias in comparative effectiveness research. Med. Care doi: /mlr (2013) (Epub ahead of print). 3 Gagne JJ, Bykov K, Willke RJ, Kahler KH, Subedi P, Schneeweiss S. Treatment dynamics of newly marketed drugs and implications for comparative effectiveness research. Value Health 16(6), (2013). Outlines some of the concerns of studying comparative effectiveness of newly marketed agents. Uses a case example to illuminate some of the major challenges. 4 Schneeweiss S, Gagne J, Glynn R, Ruhl M, Rassen J. Assessing the comparative effectiveness of newly marketed medications: methodological challenges and implications for /cer J. Comp. Eff. Res. (Epub ahead of print) future science group

7 Performance of the disease risk score in studies with selection bias Research Article drug development. Clin. Pharmacol. Ther. 90(6), (2011). 5 Reams BD, O Malley CD, Critchlow CW, Lauffenburger JC, Brookhart MA. Changing patterns of use of osteoporosis medications in the years after launch: implications for comparative effectiveness research. Pharmacoepidemiol. Drug Saf. 23(3), (2014). 6 Etminan M, Samii A. Pharmacoepidemiology I: a review of pharmacoepidemiologic study designs. Pharmacotherapy 24(8), (2004). 7 Strom BL. Pharmacoepidemiology (3rd Edition). Wiley, NY, USA (2000). 8 Sturmer T, Joshi M, Glynn RJ, Avorn J, Rothman KJ, Schneeweiss S. A review of the application of propensity score methods yielded increasing use, advantages in specific settings, but not substantially different estimates compared with conventional multivariable methods. J. Clin. Epidemiol.59(5), (2006). 9 Tadrous M, Gagne JJ, Sturmer T, Cadarette SM. Disease risk score as a confounder summary method: systematic review and recommendations. Pharmacoepidemiol. Drug Saf. 22(2), (2013). Gives an overview of the disease risk score (DRS) and its applications in the published literature. 10 Arbogast PG, Ray WA. Performance of disease risk scores, propensity scores, and traditional multivariable outcome regression in the presence of multiple confounders. Am. J. Epidemiol. 174(5), This is a seminal methods paper comparing the DRS to other means of confounder control. 11 Arbogast PG, Kaltenbach L, Ding H, Ray WA. Adjustment for multiple cardiovascular risk factors using a summary risk score. Epidemiology 19(1), (2008). 12 Cadarette SM, Gagne JJ, Solomon DH, Katz JN, Sturmer T. Confounder summary scores when comparing the effects of multiple drug exposures. Pharmacoepidemiol. Drug Saf. 19(1), 2 9 (2010). Outlines the use of confounder summary scores and explores some important issues with DRS application when using an unexposed cohort to derive the DRS. 13 Perkins SM, Tu W, Underhill MG, Zhou XH, Murray MD. The use of propensity scores in pharmacoepidemiologic research. Pharmacoepidemiol. Drug Saf. 9(2), (2000). 14 Glynn RJ, Schneeweiss S, Sturmer T. Indications for propensity scores and review of their use in pharmacoepidemiology. Basic Clin. Pharmacol. Toxicol. 98(3), (2006). 15 Rosenbaum PR, Rubin DB. The central role of the propensity score in observational studies for causal effects. Biometrika 70(1), (1983). 16 Arbogast PG, Ray WA. Use of disease risk scores in pharmacoepidemiologic studies. Stat. Methods Med. Res. 18(1), (2009). Discusses the history and use of the DRS. Outlines the construction and application of DRS methods. 17 Miettinen OS. Stratification by a multivariate confounder score. Am. J. Epidemiol. 104(6), (1976). This is the innovator paper that first proposed the use of the DRS. 18 Austin PC. An introduction to propensity score methods for reducing the effects of confounding in observational studies. Multivariate Behav. Res. 46(3), (2011). 19 D Agostino RB. Propensity scores in cardiovascular research. Circulation 115(17), (2007). 20 Pike M, Anderson J, Day N. Some insights into Miettinen s multivariate confounder score approach to case control study. Epidemiol. Community Health 33(1), (1979). 21 Francis Cook E, Goldman L. Performance of tests of significance based on stratification by a multivariate confounder score or by a propensity score. J. Clin. Epidemiol. 42(4), (1989). 22 Arbogast P, Seeger J, Group DMCSVW. Summary variables in observational research: propensity scores and disease risk scores. Effective Health Care Program 33 (2012) Cadarette SM, Gagne JJ, Solomon DH, Katz JN, Sturmer T. Confounder summary scores when comparing the effects of multiple drug exposures. Pharmacoepidemiol. Drug Saf. 19(1), 2 9 (2010). 24 Glynn RJ, Gagne JJ, Schneeweiss S. Role of disease risk scores in comparative effectiveness research with emerging therapies. Pharmacoepidemiol. Drug Saf. 21(Suppl. 2), (2012). 25 Cadarette SM, Lévesque L, Mamdani M et al. Comparison of orally administered bisphosphonate drugs in reducing the risk of hip fracture in older adults: a population-based cohort study. CMAJ Open 1(3), E97 E105 (2013). Outlines and describes the cohort that was utilized in this study. Information on the creation of the cohort and details on baseline charachterestics can be found in this article. 26 Lix LM, Azimaee M, Osman BA et al. Osteoporosis-related fracture case definitions for population-based administrative data. BMC Public Health 12, 301 (2012). 27 Juurlink D, Preyra C, Croxford R. Canadian Institute for Health Information Discharge Abstract Database: a validation study (2006) Watts NB, Chines A, Olszynski WP et al. Fracture risk remains reduced one year after discontinuation of risedronate. Osteoporos. Int. 19(3), (2008). 29 Austin PC. Comparing paired vs non-paired statistical methods of analyses when making inferences about absolute risk reductions in propensity-score matched samples. Stat. Med. 30(11), (2011). 30 Black DM, Schwartz AV, Ensrud KE et al. Effects of continuing or stopping alendronate after 5 years of treatment: the Fracture Intervention Trial Long-term Extension (FLEX): a randomized trial. JAMA 296(24), (2006). 31 Borah BJ, Moriarty JP, Crown WH, Doshi JA. Applications of propensity score methods in observational comparative future science group /cer.15.40

8 Tadrous, Mamdani, Juurlink, Krahn, Lévesque & Cadarette effectiveness and safety research: where have we come and where should we go? J. Comp. Eff. Res. 3(1), (2014). 32 Pasternak B, Svanstrom H, Ranthe MF, Melbye M, Hviid A. Atypical antipsychotics olanzapine, quetiapine, and risperidone and risk of acute major cardiovascular events in young and middle-aged adults: a nationwide register-based cohort study in denmark. CNS Drugs 28(10), (2014). 33 Wyss R, Lunt M, Brookhart MA, Glynn RJ, Stürmer T. Reducing bias amplification in the presence of unmeasured confounding through out-of-sample estimation strategies for the disease risk score. J. Causal Inference 2(2), (2014). 34 Rassen JA, Gagne J, Glynn RJ, Franklin JM, Schneeweiss S. Evaluating newly-marketed medications with a highdimensional disease risk score estimated in a historical cohort. Presented at: Pharmacoepidemiology and Drug Safety, Quebec, Canada, 26 August Schneeweiss S, Gagne JJ, Glynn RJ, Ruhl M, Rassen JA. Assessing the comparative effectiveness of newly marketed medications: methodological challenges and implications for drug development. Clin. Pharmacol. Ther. 90(6), (2011). 36 Cadarette SM, Carney G, Baek D, Gunraj N, Paterson JM, Dormuth CR. Osteoporosis medication prescribing in british columbia and ontario: impact of public drug coverage. Osteoporos. Int. 23(4), (2012). 37 Sturmer T, Schneeweiss S, Brookhart MA, Rothman KJ, Avorn J, Glynn RJ. Analytic strategies to adjust confounding using exposure propensity scores and disease risk scores: nonsteroidal antiinflammatory drugs and short-term mortality in the elderly. Am. J. Epidemiol 161(9), (2005). 38 Maclean C, Newberry S, Maglione M et al. Systematic review: comparative effectiveness of treatments to prevent fractures in men and women with low bone density or osteoporosis. Ann. Intern. Med. 148(3), (2008). 39 Cadarette SM, Katz JN, Brookhart MA, Sturmer T, Stedman MR, Solomon DH. Relative effectiveness of osteoporosis drugs for preventing nonvertebral fracture. Ann. Intern. Med. 148(9), (2008). 40 Freemantle N, Cooper C, Diez-Perez A et al. Results of indirect and mixed treatment comparison of fracture efficacy for osteoporosis treatments: a meta-. Osteoporos. Int. 24(1), (2013). 41 Franklin JM, Rassen JA, Bartels DB, Schneeweiss S. Prospective cohort studies of newly marketed medications: using covariate data to inform the design of large-scale studies. Epidemiology 25(1), (2014). 42 Hansen BB. The prognostic analogue of the propensity score. Biometrika 95(2), (2008). 43 Leacy FP, Stuart EA. On the joint use of propensity and prognostic scores in estimation of the average treatment effect on the treated: a simulation study. Stat. Med. 33(20), (2014) /cer J. Comp. Eff. Res. (Epub ahead of print) future science group