info@sentinelsystem.org 1 Using the Self-Controlled Risk Interval (SCRI) Method to Study Vaccine Safety W. Katherine Yih, PhD, MPH Sentinel/PRISM Epidemiology of Vaccine Safety ICPE pre-conference course, Aug. 27, 2017
info@sentinelsystem.org 2 Funding source: Food and Drug Administration I have no conflicts of interest
info@sentinelsystem.org 3 Self-controlled risk interval design Each individual contributes person-time in prespecified risk and control intervals ( windows ) Vaccination AE Risk interval Control interval Vaccination AE Risk interval Control interval
info@sentinelsystem.org 4 Self-controlled risk interval design Uses only vaccinated cases with the HOI in either pre-specified risk or control interval Each subject serves as own control this adjusts for fixed confounders (e.g., sex, ethnicity, SES) Lengths of risk and control intervals are fixed but needn t be equal H 0 : risk of outcome on average day in RW = risk of outcome on average day in CW
info@sentinelsystem.org 5 Self-controlled risk interval design Unadjusted RR point estimate is just as you d expect Example 1: Risk interval: Days 1-28 Control interval: Days 29-56 13 events in risk interval, 9 in control interval Unadjusted RR = 13/9 = 1.4 Example 2: Risk interval: Days 1-7 Control interval: Days 22-42 13 events in risk interval, 9 in control interval Unadjusted RR = (13*3)/9 = 4.3
info@sentinelsystem.org 6 Self-controlled risk interval design Advantages Controls for fixed potential confounders Uses only exposed cases, avoids bias affecting cohort studies when some exposed misclassified as unexposed Disadvantages Less statistical power than cohort designs that use large amount of historical or concurrent data on unexposed Any time-varying confounding must be explicitly controlled for
info@sentinelsystem.org 7 Two case studies Rotavirus vaccines (RV) and intussusception (IS) Quadrivalent HPV vaccine (Gardasil or HPV4 ) and venous thromboembolism (VTE) Data/surveillance system FDA-sponsored Sentinel Initiative (PRISM) Medical claims data from large health insurance companies 43 million people currently accruing new data (as of Jan. 2017)
info@sentinelsystem.org 8 Rotavirus vaccines & intussusception: Motivation RotaShield licensed in 1998 but withdrawn in 1999 due to risk of intussusception (1-2 excess cases/10,000) For RotaTeq (2006) and Rotarix (2008), no increased risk observed in clinical trials of >60,000 children each, but post-licensure studies in other countries suggested increased risk after both FDA requested PRISM study to determine risk among U.S. infants
info@sentinelsystem.org 9 Rotavirus vaccines & intussusception: Design Self-controlled risk interval (SCRI) (vaccinated infants only) controls for fixed risk factors Risk intervals: Days 1-7 and 1-21 Temporal scan statistics to look for clustering Confounding by age Explicitly controlled for age
Rate per 100,000 infants 0-6 DAYS 14-20 DAYS 28-34 DAYS 42-48 DAYS 56-62 DAYS 70-76 DAYS 84-90 DAYS 98-104 DAYS 112-118 DAYS 126-132 DAYS 140-146 DAYS 154-160 DAYS 168-174 DAYS info@sentinelsystem.org 10 182-188 DAYS 196-202 DAYS 210-216 DAYS 224-230 DAYS 238-244 DAYS 252-258 DAYS 266-272 DAYS 280-286 DAYS 294-300 DAYS 308-314 DAYS 322-328 DAYS 336-342 DAYS 350-356 DAYS Intussusception incidence by age (from 11 years of U.S. HCUP data when rotavirus vaccine not used) 70 60 50 40 30 20 10 0 J Tate et al. Trends in IS hospitalizations Pediatrics 2008;121(5):e1125-1132.
Rate per 100,000 infants 0-6 DAYS 14-20 DAYS 28-34 DAYS 42-48 DAYS 56-62 DAYS 70-76 DAYS 84-90 DAYS 98-104 DAYS 112-118 DAYS 126-132 DAYS 140-146 DAYS 154-160 DAYS 168-174 DAYS info@sentinelsystem.org 11 182-188 DAYS 196-202 DAYS 210-216 DAYS 224-230 DAYS 238-244 DAYS 252-258 DAYS 266-272 DAYS 280-286 DAYS 294-300 DAYS 308-314 DAYS 322-328 DAYS 336-342 DAYS 350-356 DAYS Intussusception incidence by age 70 Dose 3 (for RotaTeq) 60 Dose 2 50 40 30 Dose 1 20 10 0 J Tate et al. Trends in IS hospitalizations Pediatrics 2008;121(5):e1125-1132.
info@sentinelsystem.org 12 Adjustment for time-varying confounding Incorporate offset term into the logistic regression Offset adjusts for difference in background risk in RW and CW Each case has an offset term whose value depends on case s age at vaccination Offset = natural log (ln) of Estimated cumulative baseline risk in RW Estimated cumulative baseline risk in CW
Rate per 100,000 infants 0-6 DAYS 14-20 DAYS 28-34 DAYS 42-48 DAYS 56-62 DAYS 70-76 DAYS 84-90 DAYS 98-104 DAYS 112-118 DAYS 126-132 DAYS 140-146 DAYS 154-160 DAYS 168-174 DAYS info@sentinelsystem.org 13 182-188 DAYS 196-202 DAYS 210-216 DAYS 224-230 DAYS 238-244 DAYS 252-258 DAYS 266-272 DAYS 280-286 DAYS 294-300 DAYS 308-314 DAYS 322-328 DAYS 336-342 DAYS 350-356 DAYS Background risk of intussusception in risk and control intervals for typically timed Doses 1 & 3 70 60 50 40 30 20 10 0
info@sentinelsystem.org 14 Rotavirus vaccines & intussusception: RotaTeq results 507,874 Dose 1; 1,277,556 total doses Dose 1 associated with increased risk of intussusception in the 1-7 and 1-21 days after vaccination Statistically significant cluster found on Days 3-7 after vaccination (Dose 1 and all doses combined) 3 2 1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 All attributable risk point estimates in range of 1.1-1.5 excess cases per 100,000 first doses ( 1/10 of risk associated with RotaShield)
info@sentinelsystem.org 15 Critiques of our RV-IS age adjustment 1. HCUP population different from PRISM population, based on hospital discharge data, etc. So HCUP age-specific incidence estimates could differ from true age-specific incidence in PRISM population Response: HCUP estimates based on 3,463 cases, stable As long as curve correct in relative sense (x times higher or lower than HCUP curve), offset terms correct Conducted post hoc robustness analysis using modeled risk of intussusception in unexposed person-time of study population (with 97 confirmed cases)
Excess cases per 100,000 vaccinees RotaTeq attributable risks for Days 1-7 RW, by dose and age adjustment method 3 Dose 1 Dose 2 Dose 3 2.5 2 1.5 1 0.5 0-0.5 info@sentinelsystem.org 16
info@sentinelsystem.org 17 Critiques of our RV-IS age adjustment 2. Uncertainty in estimating age-specific background incidence not taken into account, rather incidence treated as known without error Variance of rotavirus-intussusception RRs and ARs underestimated, confidence intervals too narrow
info@sentinelsystem.org 18 Critiques of our RV-IS age adjustment 2. Uncertainty in estimating age-specific background incidence not taken into account, rather incidence treated as known without error Variance of rotavirus-intussusception RRs and ARs underestimated, confidence intervals too narrow Response (after rotavirus-intussusception study over): Random adjustment method developed by M. Kulldorff for PRISM study of influenza vaccine and febrile seizures; accounts for uncertainty in baseline risk estimates L. Li conducted simulation study comparing performance of fixed adjustment and newer random adjustment
info@sentinelsystem.org 19 Comparison of fixed adjustment and random adjustment (Lingling Li, 2015 ms.) (Random adjustment takes into account uncertainty in estimating age-specific background incidence) Random adjustment performs well in general Fixed adjustment has comparable performance if no. in baseline data (n b ) no. in study (n s ) Rotavirus study met this condition: n b = 97, n s 30
info@sentinelsystem.org 20 Gardasil & venous thromboembolism: Motivation Signal from VAERS, although 90% of the 31 had a known VTE risk factor (Slade et al., JAMA 2009) Signal from VSD, although all 5 had a known VTE risk factor (Gee et al., Vaccine 2011) FDA s Pediatric Advisory Committee requested PRISM study
info@sentinelsystem.org 21 Gardasil & venous thromboembolism: Design Self-controlled risk interval (SCRI) (vaccinated infants only) controls for fixed risk factors Risk intervals: Days 1-7 and 1-28 Temporal scan statistics to look for clustering Confounding by oral contraceptive (CHC) use Explicitly controlled for duration of CHC use
VTE risk by duration of combined hormonal contraceptive (CHC) use Vlieg et al. BMJ 2009;339:b2921 22
Age-adjusted odds ratio VTE risk by duration of combined hormonal contraceptive (CHC) use 25 20 15 HPV4 risk control If HPV4 administered soon after contraceptive initiation, then VTE risk in risk interval could be higher than in control interval biases toward association between HPV4 and VTE 10 5 0 No OC (reference) Vlieg et al. BMJ 2009;339:b2921. 3 >3 and 6 >6 and 12 >12 and 24 >24 and 60 >60 Duration of use (months) 23
To adjust HPV4-VTE SCRI analysis Need to characterize VTE risk with changing duration of CHC use, with greater granularity than this: Vlieg et al. BMJ 2009;339:b2921 info@sentinelsystem.org 24 24
To adjust HPV4-VTE SCRI analysis Use source population (in same time period plus from < 2006 if available) Identify CHC use National Drug Codes in claims data Generic names shared with Data Partners to add missing/homegrown codes Determine length of time on CHCs as of outcome Using Poisson regression, model risk of VTE by number of weeks on CHCs (adjusting for other relevant covariates, e.g., Data Partner, age, secular trends) info@sentinelsystem.org 25 25
CHC-VTE modeling to adjust HPV4-VTE SCRI analysis Included in modeling: ~9,000 potential cases of VTE ~12 million person-years Model adjusted for Data Partner, age, whether and for how long on CHCs, estrogen dosage, and secular trends Point after CHC initiation at which VTE risk levels out: Risk during 9-<12 mo. risk at 12 mo. So considered VTE risk to plateau after 9.0 mo. on CHCs info@sentinelsystem.org 26 26
Predicted VTE incidence by week after CHC initiation Week after combined hormonal contraceptive initiation 27 info@sentinelsystem.org 27
info@sentinelsystem.org 28 Adjustment for time-varying confounding Incorporate offset term into the logistic regression Offset adjusts for difference in background risk in RW and CW Each case has an offset term whose value depends on case s age duration of CHC use at vaccination Offset = natural log (ln) of Estimated cumulative baseline risk in RW Estimated cumulative baseline risk in CW
HPV4-VTE SCRI analyses without and with adjustment for CHC duration A. Analyses with all definite VTE cases, with no adjustment for CHC use Dose Days in RW Cases in RW Cases in CW RR 95% CI lower bound 95% CI upper bound 1 1-28 4 5 0.60 0.15 2.27 2 1-28 4 8 0.50 0.13 1.59 3 1-28 5 4 1.25 0.33 5.05 All 1-28 13 17 0.70 0.33 1.44 1 1-7 0 5 0 -- -- 2 1-7 1 8 0.50 0.03 2.73 3 1-7 1 4 1.00 0.05 6.76 All 1-7 2 17 0.43 0.07 1.51 B. Analyses with all definite VTE cases, with adjustment for CHC use Dose Days in RW Cases in RW Cases in CW RR 95% CI lower bound 95% CI upper bound 1 1-28 4 5 0.61 0.15 2.32 2 1-28 4 8 0.47 0.13 1.50 3 1-28 5 4 1.29 0.34 5.21 All 1-28 13 17 0.70 0.33 1.43 1 1-7 0 5 0 -- -- 2 1-7 1 8 0.47 0.03 2.55 3 1-7 1 4 1.09 0.06 7.38 All 1-7 2 17 0.43 0.07 1.50 29
info@sentinelsystem.org 30 Gardasil & venous thromboembolism: Results 1,423,399 total doses No evidence of increased risk of VTE after Gardasil SCRI analyses Temporal scan analyses
Summary of the two case studies RV5 (RotaTeq) & intussusception (IS) (> 1.2 M doses) Design SCRI SCRI HPV4 (Gardasil) & venous thromboembolism (VTE) (> 1.4 M doses) Risk intervals Days 1-7, 1-21 Days 1-7, 1-28 Control intervals Days 22-42 Days 36-56 (Dose 1) Days 36-63 (Doses 2 & 3) Time-varying confounder Age Duration of CHC (contraceptive) use Method(s) of adjustment Multiple, including using curve from external data treated as known with certainty 1) none, 2) using curve from internal data treated as known with certainty Findings Slightly increased risk of IS No increased risk of VTE Policy implications Conclusions re. timevarying confounding Label change, no change in ACIP rec s Adjustment crucial, but method didn t matter No change in label or ACIP rec s Adjustment didn t matter 31
info@sentinelsystem.org 32 Conclusions Self-controlled risk interval design often a good choice for vaccine safety studies Controls for fixed potential confounding But power can be an issue when outcome is rare and/or effect size modest Control for time-varying confounding can be implemented in several ways Random adjustment appropriate where internal data reasonably plentiful and individual-level data available Otherwise, fixed adjustment (using either internal or comparable external data) is fine, as long as no. in baseline sample no. in SCRI sample (RW+CW)
info@sentinelsystem.org 33 Papers about the two PRISM studies Yih WK, Lieu TA, Kulldorff M, Martin D, McMahill-Walraven CN, Platt R, Selvam N, Selvan M, Lee GM, Nguyen M. Intussusception risk after rotavirus vaccination in U.S. infants. N Engl J Med. 2014;370:503-12. Yih WK, Greene SK, Zichittella L, Kulldorff M, Baker MA, de Jong JLO, Gil-Prieto R, Griffin MR, Jin R, Lin ND, McMahill- Walraven CN, Reidy M, Selvam N, Selvan MS, Nguyen MD. Evaluation of the risk of venous thromboembolism after quadrivalent human papillomavirus vaccination among US females. Vaccine. 2016;34:172-8.