Tuning Epidemiological Study Design Methods for Exploratory Data Analysis in Real World Data

Transcription

1 Tuning Epidemiological Study Design Methods for Exploratory Data Analysis in Real World Data Andrew Bate Senior Director, Epidemiology Group Lead, Analytics 15th Annual Meeting of the International Society of Pharmacovigilance, Prague, 30 October 2015

2 Disclosures and Acknowledgement I am a full time employee of Pfizer and hold stocks and stock options The work presented here represent a component of a multi-year internal to Pfizer methodology programme to test and better understand Active Surveillance approaches thanking the following for their contributions to the production of these results Xiaofeng Zhou, Kathy Liu, Sundaresan Murugesan, Rongjun Shen, Richard Gong and Bing Cai Desvenlafaxine (Pristiq) used for this study is a Pfizer product 2

3 Overview The importance of assessing signal detection capability in longitudinal observational data Current knowledge about signal capability in such data Read out the results of quantitative signal detection testing on method testing focussing on Adapting and assessing standard epidemiological approaches for signal detection Test set of more recently marketed products Focussed detailed analysis of pre-defined small set of drugoutcome pairs 3

4 Increasingly Important to Harness the Power of Real World Evidence for Safety Characterize Patient Risk Profile Standing Cohorts Approval Evaluate Product Risks Active Surveillance Monitor and detect signals in defined patient cohorts using innovative analytic methods Post Approval Safety Studies Compare medication risks in the real world, as prescribed and taken during routine clinical practice EMRs Claims Registries Risk Minimization Evaluate the effectiveness of risk minimization measures (e.g., product label/education) 4

5 Good (but Imperfect) Method Performance Against Well Established Safety Reference Set Statistical Method Lower 95% CI >1 UK EMR THIN Median across OMOP lab of 10 RWA databases Sensitivity Specificity Sensitivity Specificity PRR USCCS HDPS Comparison to OMOP reference set of 53 well studied drug outcome pairs of established drugs showed no single best method Generalizability to NMEs? USCCS and HDPS are two very different analytic approaches extensively used by the epidemiological community but with clearly different designs might be more effective for different drug-event pairs Evaluated extensively for observational studies PRR is a transparent approach and a key signal detection method for spontaneous reports - ideal reference method 5

6 Approach to Gain Insights Into Signal Detection Utility in Longitudinal Observational Data Resources for More Recently Marketed Products Drugs of Focus 5 Outcomes of Interest Databases UK EMR US Claims Active Surveillance Algorithms Cumulative Adverse Events (angioedema) Lisinopril Versus Unexposed Months Observed AEs Expected AEs RR Relative Risk Focused analysis Inform understanding of signal detection method performance for recently marketed products More understanding on utility of different algorithms for different kinds of outcomes More clarity on how signal detection on such data can add value compared to signals generated from other data sources externally 6

7 Reference Health Outcomes of Interest Used for Method Testing The study focused on ability of methods to detect labelled outcomes Selected outcomes For Pristiq Hypertension, Orthostatic hypotension, Proteinuria, Hyperlipidemia, All Fractures For Humira Acute Myocardial Infarction (AMI), GI Perforation, Herpes Zoster, Interstitial Lung Disease, Lymphoma, Pneumonia A priori had varying levels of confidence in how well suited the databases will be for signal detection of these outcomes with different methods Examining the variability - an important study component 7

8 Epidemiologic Design and Statistical Methods Implementation for Primary Analysis Choice of comparator Case definition Risk period PRR USCCS HDPS All other drugs N/A Comparator for Humira: Other biologics, Comparator for Pris<q : An<depressant, specifically Cymbalta (THIN): used Lexapro as study drug 1st occurrence of the outcome of interest Exposure dura<on plus 30 days from the end of the exposure 1st occurrence of the outcome of interest Exposure dura<on plus 30 days from the end of the exposure 1st occurrence of the outcome of interest Ever aler first prescrip<on of study drug or comparison drug Drug: defined by diagnoses codes NDC codes in Optum and multilexid in THIN. Indication not considered explicitly, but implicit in choice of comparator HDPS: 5 dimensions (prescription, inpatient procedures, outpatient procedure, inpatient diagnoses, outpatient diagnoses) for Optum database; 3 dimensions (prescription, procedure and diagnoses) for THIN database. Data until end of

9 Details of Statistical Method Testing for USCCS USCCS model considers one drug and one AE which is assumed arising from a non-homogeneous Poisson process Event X Drug A A X It automatically controls for time-fixed covariates It only includes cases in the analysis which are then used as their own controls. The USCCS conditional likelihood Key assumption: conditionally independent events and events conditionally independent of exposure 9

10 Details of Statistical Method Testing for HDPS PS Estimation Event X PS Estimation Drug A Event X Drug B Create cohorts of patients who use drug A or drug B Estimate Propensity Score (PS) for a given exposure for each individual based on multiple variables that might be potential confounders (e.g. age, gender, other therapy use) Risk Estimation then adjusted for confounding using Propensity Scores PS-based adjustment done with different methods including logistic regression or SMR weighted method 10

11 Results for Method Testing on Humira Data Several, but Not All, Pairs Highlighted Humira in Optum AMI GI Perforation Herpes Zoster Interstitial Lung Disease Lymphoma Pneumonia PRR 1.01 (0.85) 1.33 (1.25) 1.97 (1.80) 0.59 (0.57) 0.89 (0.70) 0.85 (0.79) SCCS 1.12(0.87) 0.78(0.71) 1.41(1.23) 0.69(0.66) 1.43(1.02) 1.15(1.04) HDPS 0.97(0.83) 1.22(1.14) 1.12(1.02) 0.99(0.95) 0.82(0.66) 1.00(0.94) Note: Results shown are point estimate with lower bound of 95% confidence interval (LBCI) in the parentheses. Highlighted in red are those with LBCI greater than one. 11

12 Results for Method Testing on Antidepressant Data Few Pairs Highlighted Pristiq in Optum Lexapro in THIN Fracture Hyperlipidemia Hypertension Orthostatic Hypotension Proteinuria Fracture Hyperlipidemia Hypertension Orthostatic Hypotension Proteinuria PRR 0.50(0.47) 0.37(0.36) 0.37(0.35) 0.81(0.72) 0.46(0.40) 0.21(0.20) 0.31(0.30) 0.19(0.18) 0.49(0.42) 0.30(0.25) USCCS 0.96(0.89) 0.40(0.39) 0.40(0.37) 0.97(0.83) 0.98(0.80) 1.22(1.14) 1.03(0.94) 0.60(0.56) 1.51(1.22) 1.32(1.06) HDPS 0.70(0.67) 0.66(0.62) 0.62(0.61) 0.59(0.53) 0.63(0.54) 1.47(1.35) 1.87(1.67) 1.47(1.33) 1.37(1.11) 1.53(1.24) Note: Point estimates shown with lower bound of 95% confidence interval (LBCI) in parentheses LBCI scores greater than 1 are highlighted in red 12

13 Secondary Analyses Additional analysis performed for SCCS and HDPS Varied risk period selections Varied criteria of case selections (or inclusion criteria) Examined variation across the two heterogeneous database by also conducting analysis of Lexapro in Optum data for direct comparison to Lexapro versus Cymbalta in THIN For HDPS Compared Propensity Score contribution by changing covariates in model, risk estimation method and also reviewed crude estimates In depth consideration of approaches to PS matching and underlying PS distribution assessment (HDPS) For SCCS Examined capability of monitoring change in estimates over time 13

14 Outcome Criterion change for SCCS Increases Estimates Consistently Pristiq- Optum Humira- Optum Lexapro -THIN All outcomes highlighted for Humira with incident case Same trend across study pairs - fractures, hypotension and proteinuria highlighted, hypertension and hyperlipideamia less so Very similar pattern for Lexapro in Optum data (results not shown) By contrast varying time at risk period had limited impact New case is the subset of all cases with at least 12 months enrollment on or before the 1 st outcome 14

15 HDPS Adjusted Odds Ratio for Antidepressants in Optum and THIN Pristiq- Optum Lexapro -THIN Lexapro -Optum When varying adjustment approach, adjusting for demographics and propensity score, performed best and regression approach highlighted more SDRs than SMR weighting New case after the first exposure clearly more effective at highlighting SDRs (results not shown) Some indication that in Optum data that restricted to outcomes within first 30 days of first exposure improved performance further, not observed in THIN data (result not shown) 15

16 HDPS Adjusted Odds Ratio for Humira in Optum While adjustment for demographics led to higher scores than unadjusted estimates, when comparing Humira to all bdmards the benefit of SMR or regression based HDPS was less clear Possibly due to heterogeneity across a drug class as comparator (general lack of research on drug class as comparator when using HDPS) 16

17 Propensity Score Distributions Similar But Not Identical Distribution of propensity score of the group exposed to Lexapro in the Optum database. Distribution of propensity score of group unexposed to Lexapro (comparator group) in the Optum database. Some differences observed in Odds Ratio estimates with HDPS when varying approach for using PS distributions o Specifically logistic regression compared to SMR weighting method Ø Such differences can be indicative of non-balanced PS distributions 17

18 US Claims Data Also Potentially Useful for Identifying Signals Earlier: Optum Data Relative Incidence with 95% LBCI Q Q3 2003Q4 Results of Self Controlled Case Series (SCCS) Q1 2004Q2 2004Q Q4 2005Q Q2 2005Q Q Q1 2006Q2 Generally observed flat trends over time for most drug- adverse event pairs 2006Q Herpes zoster reported in clinical trials for Humira Analyses of real world data identified herpes zoster as an issue post-marketing requiring further investigation 2006Q4 2007Q1 2007Q2 2007Q3 2007Q4 2008Q1 2008Q2 2008Q3 2008Q4 2009Q1 2009Q2 2009Q3 2009Q4 2010Q1 2010Q2 2010Q3 2010Q4 LBCI=Lower Bound Confidence Interval 18

19 Limitations In US Insurance claims and UK EMRs, solely defining outcome by diagnosis code (ICD9 or Read Code) is not always accurate USCCS: Method can be volatile to selection of risk period Necessarily risk window selection and confounder adjustment approaches will be suboptimal in a signal detection framework, as generic parameter selection needed that are effective for large numbers of drug-outcome pairs (e.g. acute v chronic outcomes) HDPS: comparator group selection is challenging: methods highlight SDRs if difference between drug of interest and comparison drug/group. 19

20 References Bate A, Evans SJW. Quantitative signal detection using spontaneous ADR reporting. Pharmacoepidemiology and Drug Safety (6): pp Cai B, Liu Q, Geier J, Bate A. An Algorithm To Predict Biologic DMARD Use in the THIN Database. Pharmacoepidemiology and Drug Safety 22: S391 Cai B, Murugesan S, Geier J, Bate A Applying High Dimensional Propensity Score (HDPS) in a Exploratory Data Analysis with a US Claims Database for Recent Medicinal Products. Pharmacoepidemiology and Drug Safety 23: S13 Schneeweiss S, Rassen JA, Glynn RJ, Avorn J, Mogun H, & Brookhart MA. (2009). High-dimensional propensity score adjustment in studies of treatment effects using health care claims data. Epidemiology 20(4), 512. Whitaker HJ, Hocine MN, & Farrington CP. (2008). The methodology of self-controlled case series studies. Statistical methods in medical research. 18(1) Zhou X, Cai B, Murugesan S, et al. An Application of Univariate Self Case Control Series Method Using THIN Database in the OMOP CDM for Active Drug Safety Surveillance. Pharmacoepidemiology and Drug Safety 21: S283 Zhou X, Murugesan S, Bhullar H, Liu Q, Cai B, Wentworth C, Bate A An Evaluation of the THIN Database in OMOP Common Data Model for Active Drug Safety Surveillance Drug Safety. 36(2) pp: Zhou X, Shen R, Geier J, Bate A Adapting and Evaluating Self-Controlled Case Series Method (SCCS) for Signal Screening of a Recently Marketed Drug Using a US Claims Database Pharmacoepidemiology and Drug Safety 23: S181 20

21 Discussion For well established products and outcomes previous research shows that signals can be detected, but with imperfect performance Our work suggests a similar position for more recently marketed products and the potential value for signal detection in observational databases No single approach, nor set of design choices, performed uniformly better Volatility and some unpredictable variation in observed scores emphasized challenges for routine signal detection in such data Use of a range of approaches is therefore likely to be important in signal detection (e.g. for acute and chronic outcomes) Even more so than spontaneous reports, because of the complexity and heterogeneity of underlying data, and longitudinal nature of data sets 21