Time series disturbance detection for hypothesis-free signal detection in longitudinal observational databases Andrew Bate Senior Director, Epidemiology Group Lead, Analytics On behalf of Manfred Hauben, Ed Whalen, Andrew Bate ISOP Annual Conference 216 Agra 17 October 216
Disclosures I am a full time employee of Pfizer and hold stocks and stock options
Background: Visualization Insights from Chronograph In a UK Longitudinal Observational Database (EMRs) IC 5-5 Omeprazole - Acute Pancreatitis -3-2 -1 1 2 3 Information Component (IC)* shows unexpected recording of outcomes relative to time of prescription Number of events 6 4 2 Observed Expected -3-2 -1 1 2 3 Months relative to prescription Spontaneous reports valuable, but give limited insights in such situations * IC is a Bayesian shrinkage observed-toexpected ratio on a logarithmic scale
Objectives Signal detection in longitudinal observational databases (LODs) is an emerging focus of research LODs provide a rich and complex data source for signal detection The chronograph visualization tool shows changes in the IC disproportionality measure over time for specific events stratified into monthly pre/post exposure windows Currently interestingness of chronographs is clinical review based primarily on subjective visual inspection/heuristics Subjectivity motivates a search for chronograph patterns that 1. have sufficient positive predictive value and 2. are amenable to annotation and automated implementation Goal: Develop an automated triage that reduces various chronograph patterns of interest into qualitative summaries, allowing clinicians to focus better on potential signals
Categories of outliers for time series disturbance Innovational outlier (IO) initial impact with effects lingering over subsequent observations Additive outlier (AO) Surprisingly large or small value occurring for a single observation. Subsequent observations are unaffected by an additive outlier. Level shift (LS) All observations appearing after the outlier move to a new level ie permanent effect Temporary Change (TC) Observations after the outlier all seem to be at a new level but the effect of the outlier diminishes exponentially over subsequent observations: eventually, the series returning to normal Ref Chen C, Liu LM. Joint estimation of model parameters and outlier effects in time series. Journal of the American Statistical Association. 1993 Mar 1;88(421):284-97
Method In Noren et al 21 two visually distinct true positives of labelled ADRs for nifedipine were presented in UK EMR IMS Disease Analyzer Transient (flushing) Persistent (swelling) Scanned for time series disturbance on these two examples (Also on a consistent pattern with no visual clear change i.e. true negative result not shown) Unclear which metric in chronograph to best monitor for perturbations. Tested on IC width of CI and gamma distribution Showing results here for IC value - similar results obtained for the other two metrics Flushing - Swelling
Nifedipine Flushing in UK THIN EMR Grey curve represents chronograph, blue curve chronograph adjusted accounting for detected time series disturbances Additive Outlier detected at 28, likely spurious Temporal Change at t=1 Expected for this drug-outcome pair IC -1 1 2 3 AO TC True positive finding -3-2 -1 1 2 3 Time (months) Pfizer Confidential 7
Nifedipine Swelling in UK THIN EMR Grey curve represents chronograph, blue curve chronograph adjusted accounting for detected time series disturbances Level shifts at t= 1,2 Anticipated LS for this labelled, persistent effect so true positive IC Perhaps surprising to see 2 LSs perhaps nonspecific timing of outcome recording, or outcome onset..5 1. LS LS -3-2 -1 1 2 3 Time (months) Pfizer Confidential 8
Conclusion A novel approach for outlier detection on chronographs on LOD successfully detected different outlier types for two chronographs of well-established ADRs that were clearly different from visual review While wider testing is needed to fully understand performance, including the associated false positive and negative burdens, the results show the potential of the approach to increase efficiencies in signal detection in longitudinal observational databases Our approach was tested on chronographs, but may well apply to other visualization tools involving time trends and temporal anomaly detection Future considerations are whether this approach captures potential signals escaping visual inspection, and does outlier detection promote consistency of detection in a routine clinical review process As well as scalability testing to show tractability in routine operation