Joseph G. Mathew PGY-3 Anatomical Pathology. Michael Bonert, Asghar Naqvi McMaster University Hamilton, Ontario, CANADA

Transcription

1 Next Generation Quality: The automatic extraction and categorization of 22,760 stomach biopsy specimen parts from 110,970 free text pathology reports to assess Helicobacter pylori diagnostic rates Joseph G. Mathew PGY-3 Anatomical Pathology Michael Bonert, Asghar Naqvi McMaster University Hamilton, Ontario, CANADA

2 Disclosures No financial conflicts of interest Dr. Bonert wrote the computer code used for this study

3 Overview Objective of Study Next Generation Quality Rationale for studying Helicobacter pylori Choice of analysis/funnel plots Methods Results Implications

4 Objective Next Generation Quality: To look at the physician factor in quality of care and use clinical data to drive improvements

5 Next Generation Quality Model Continuous quality improvement using objective clinical data to provide feedback to improve quality (Luttman, 1998) In pathology, the feedback loop (from a process perspective) has been relatively weak pathologists generally do not know their diagnostic call rates

6 Our Application of Next Generation Quality Traditionally, quality assurance in pathology interpretation has focused on random audits of a limited number of cases by an expert or a panel of experts Alternative approach systematically analyze a large number of pathology reports using a custom built automated system to assess the diagnostic call rates of individual pathologists over time Allows investigation of large sample sizes to detect bias in call rates between pathologists and potentially identify areas in need of improvement

7 Rationale for Investigating Helicobacter Pylori Common pathology specimen Large sample size allows comparison of diagnostic rates with minimization of sampling bias Diagnostic error is associated with high morbidity FALSE NEGATIVES H. Pylori associated gastritis, duodenal/gastric ulcers, MALT-lymphoma, gastric adenocarcinoma FALSE POSITIVES Unnecessary treatment (triple therapy) and follow-up; possible consequences: increasing antibiotic resistance, antibiotic-associated complications Wide variation in upfront use of ancillary stains among pathologists despite lack of evidence Upfront use of special stains is not recommended by the Rodger C Haggitt Gastrointestinal Pathology Society practice guidelines Muto et al., 1975; Winawer et al., 2006; Kristjansdottir et al., 2010; Mahajan et al., 2013; Coleman et al., 2015 *images from Librepathology.com

8 Funnel Plots and Expected Variation Easy to interpret large amounts of data and identify outliers Range expected due to sampling (REDS) is larger for small sample sizes funnel shape Hypothesis generating does not determine the ideal rate, but can assess consistency Ex. Used to compare C-section rates between hospitals Bragg et al., BMJ Oct 6; 341

9 Methods: Custom Software/Data Extraction Ethics approval obtained A custom-built software program was created to analyze Laboratory Information System (LIS) text and categorize diagnoses from free text pathology reports using string matching, approximate string matching and hierarchical pruning Results assessed for accuracy by three assessors (n=200) H. pylori call rate stratified by pathologist and graphed on a funnel plot to assess diagnostic variation between pathologists

10 Results 6 years of surgical pathology reports from January 1 st 2010 to December 31 st 2016 (n = 110,970) 22,760 stomach biopsy specimens 16 pathologists read 22,176 stomach specimen parts, each with >200 specimens (between 218 and 2546 each over the study period) >99% were able to be automatically categorized >98% computer accuracy based on assessor comparison (n=200)

11 Funnel Plot: H. Pylori diagnostic rate by pathologist Summary Statistics Mean: 9.1% Median: 9.0 % Standard Deviation: 1.8% Range: % Median diagnostic rate o Pathologists using upfront special stains x Pathologists not using upfront special stains Number of specimens interpreted

12 Funnel Plot: H. Pylori diagnostic rate by pathologist Twelve of sixteen pathologists had HP diagnostic rates within the 95% confidence interval One pathologist had a diagnostic rate outside the 99.8% confidence interval (p<0.001) Two pathologists not using upfront special stains had diagnostic rates within the expected range 99.8% CI (p=0.001) 95% CI (p=0.05) o Pathologists using upfront special stains x Pathologists not using upfront special stains Number of specimens interpreted

13 Data Summary There is little variation in call rates for H. Pylori among pathologists at our institution, confirmed over a large sample size The rate of H. Pylori at our institution is within the expected range reported in the literature (5-20%) One pathologist identified as having H. pylori call rate outside the 99.8% confidence interval suggesting interpretive bias Pathologists not using upfront special stains had H. pylori diagnostic rates within the expected range findings support the Gastrointestinal Pathology society consensus statement that routine ordering of upfront special stains for H. pylori diagnosis is unnecessary

14 Implications Automatic extraction and categorization of diagnostic information from free text pathology reports can be used to monitor pathologist diagnostic rates using large samples to minimize sampling bias Population based observational data can compliment traditional interobserver variability studies/random audits by identifying diagnostic trends and areas possibly requiring further education and training This technique may represent a potential large scale data extraction tool that can be used as part of comprehensive data driven Quality Management/Improvement programs in order to recognize diagnostic trends and potential need for improvement with aim to reduce variability Methods used in this study are applicable to other diagnostic entities

15 Acknowledgments Dr. Becky Lin (University of Toronto, Department of Statistical Science) Dr. Jennifer Dmetrichuk (Pathology Resident, McMaster University) Images from Wikimedia commons unless otherwise stated

16 Choice of Analysis: Population-Based Approach Pathologists compared on the basis of the population they serve Rationale: Population-based approach is considered the norm in clinical medicine e.g. C-section rate, post-op complications, post-op infections, lab test ordering Assumptions: 1) Population disease characteristics are stable with time 2) No patient selection/referral bias