Data Science Reduces Anatomic Pathology Reporting Errors

Transcription

1 Data Science Reduces Anatomic Pathology Reporting Errors Session # 270, February 14, 2019 Jay J. Ye, MD, PhD, Pathologist Dahl-Chase Pathology Associates 1

2 Conflict of Interest Jay J. Ye, MD, PhD Has no real or apparent conflicts of interest to report. 2

3 Agenda Introduction: - Pathology process: specimens to reports - Types of errors: interpretation vs reporting - Data science: why effective Catching the errors: - Two programs: web application / auto- - Mechanisms Bigram approach Implications 3

4 Learning Objectives Define pathology reporting errors and describe their characteristics Describe data science and tools of data science Describe how data science is used to reduce reporting errors 4

5 The process of anatomic pathology: from specimens to reports Accessioning: entering demographic and specimen info Gross examination, (sectioning), and submitting of specimens: specimens to cassettes Histology: tissue in cassettes to glass slides Interpretation: from slides to final reports 5

6 Example of a report 6

7 Interpretation errors vs reporting errors Interpretation errors Professional judgement Reporting errors Not professional judgement 7

8 Examples of interpretation errors Benign breast lesion vs. breast cancer Benign mole vs melanoma Positive vs negative margin Missing the lesion 8

9 Examples of reporting errors Patient ID, left vs right (not included) Typo / voice recognition errors Internal inconsistency within the report Paraffin block designation errors Unintentional omission of special studies (immuno, special, etc.) Incorrect patient sex, etc 9

10 To reduce interpretation errors Professional training Subspecialization Policies and procedures, including certain mandatory reviews. 10

11 Characteristics of reporting errors Not related to the professional abilities Sometimes difficult to catch Generally obvious after being pointed out 11

12 Harmful effects of reporting errors Annoying the clinicians Conveying incorrect information Reducing the trustworthiness of the report Compliance issue (special studies performed, billed but not reported) 12

13 Data science and its tools 1. Data science extracts: Lots of data concise knowledge 2. Tools: R and Python, etc. Data, data, and lots of data Data data Data data Data, More data knowledge 13

14 Pairing data science with reporting error detection Data science: great at extracting knowledge from large quantity of data (including finding a needle in a hay stack) Reporting errors: Rare for each particular error, therefore sometimes difficult to catch (a needle-in-a-hay-stack problem) using data science tools to catch errors 14

15 Two programs for error catching: preliminary and final reports Report checker for PA: a web application for pathologist assistants to check for the errors in the preliminary reports. notification to pathologists: for pathologists on special studies not reported in the final reports. 15

16 Two ways to interact with data 16

17 Report Checker 17

18 Report checker Internal web address: cronos:

19 Voice Recognition Errors in Clinical Information 19

20 Voice Recognition Errors in Clinical Information: Sounds Like Specimen list / clinical information discrepancy due to phonetic similarity: lid/lip chin/shin thigh/side thigh/thyroid 20

21 Examples of voice errors in gross with a 1.2 x 0.3 sodium segment of a 0.7 x 0.4 x 0.1 tan tan-gray skin shave right hemicolectomy consisting of 9.5 cm maternal ileum, 37.5 cm of cecum 21

22 Block Designation Errors 1A 2A 3A 4A-4B 22

23 Block error 1: Omission Part 3. The largest fragment is inked at the base, trisected 3A. ** Missing 3B,3C,3D 1A lateral margin, perpendicular. 1B-1L central sections each to 1K medial margin, perpendicular **missing 1M 23

24 Block error 2: addition Part 1. The tissue is poured into a specimen bag. Intact 1A-1C Should have been 1A-1B (1C does not exist) Part 6. entirely serially submitted as 6A-6C with the tips, perpendicular within 6A and 6D 6D should have been 6C (6D does not exist) 24

25 Block error3: ambiguity 1B-1C lesion entirely submitted 1C-1D representative fragmented sections 1C-1D lesion entirely submitted. 1F nearest radial circumferential 1F representative section of mesenteric margin 1G-1H proximal polypectomy site. 25

26 Wrong Sex Name Sex Female Male report 26

27 Automatic s to pathologist Examine reports finalized by pathologists once every 5 minutes for: Additional studies performed but not reported the pathologist when such cases are found 27

28 s sent and changes made Subject: Stains not reported: S-17-##### Keratin AE1/AE3 Nothing mentioned Added: Cytokeratin AE1/3 confirms the presence of invasive carcinoma extending to the inferior margin. Subject: Stains not reported: S-18-##### CYCLIN D-1 KAPPA-ISH LAMBDA-ISH CD138 highlights plasma cells which are polyclonal on K/L ISH. CD138 highlights plasma cells which are polyclonal on kappa/lambda ISH. Cyclin D-1 is negative. 28

29 Sometimes it catches errors Subject: Stains not reported: S-17- ##### CD30 CD20 positive, PAX-5 positive B cells which also express BCL-6 and BCL-2 (dim). The atypical cells are negative for CD5, CD10, CD38, cyclin D1 and EBVish. MUM1 shows borderline positivity in a subset of the cells. Ki-67 shows a proliferation index of approximately 80%. CD3 stains (1 out of 12 stains) 29

30 Prevalence of errors by types Voice: 7-8% Blocks: 1% Stains not reported, average 1 a day Sex error: rare 30

31 Mechanisms for error catching Example of SQL and data in a table 31

32 Query: SQL wrapped in R 32

33 Processing Retrieved Data: R code 33

34 Basis for error identification 1. Error patterns: reading the text - Exact texts: e.g. Pass pending (Path pending) - Patterns of texts: e.g segment of colon (no unit, cm? mm?) 2. Conflicting information: needing information from different sources Patient sex, block submission, special studies not reported, wrong providers, etc.. 34

35 Detecting errors by exact matching 1. List items in a.txt file epidermal edema Pass pending (The correct ones should be dermal edema, path pending) 2. Read the.txt file into R, assigned to a variable Clinicalerrors : Clinicalerrors <- epidermal edema Passed pending 35

36 Detecting errors by exact matching 3. Identify any report with clinical information containing these words: clinicalvre2 <- map_chr(clinical_rpt$report, ~ paste(str_match_all(., clinicalerrors), collapse = "/")) 36

37 Users can teach the program User can teach the program to catch more and more errors over time by adding new entries to a.txt file 37

38 Detecting error by regular expression Regular expression can be used to match patterns of texts: [0-9]{1,2}[.][0-9] by no_cm <- str_match(block_rpt$report, " x [0-9]{1,2}[.][0-9] [(a-bd-z)][(a-ln-z)]([a-za- Z,. ]{5})?")[,1] Patterns cannot be easily added to the.txt file by the users 38

39 Conflicting information Examples of checking by inconsistency: Block checking: blocks submitted in database vs information parsed from gross description text Sex checking: sex in database vs naming convention and report text Stains not reported: stains billed in database vs info parsed from the diagnosis text 39

40 Stains not reported as an example: 1. Stains retrieved from the database: CD3 / ER Quant / P63-Double Stain / S100-Brown /CYCLIN D-1 2. A.txt file, each line converting a stain to the possible wordings in the reports: CYCLIN D-1,CYCLIN D1 cyclin d1 cyclin D1 Cyclin D1 cyclin D-1 Cyclin D-1 Cyclin- D1 cyclind-1 cyclind-1 cyclind1 cyclin-d1 40

41 Newly implemented bigram approach - word bigrams For the sentence: Below you will find some important instructions about HIMSS19 session information, registration, hotel and travel. Word bigrams: Below you / you will / will find / find some / registration hotel / hotel and / and travel 41

42 Construction of normal bigrams 1. Retrieving gross description text for 200k specimens. 2. Preprocessing these texts: - removing patient info, specimen label, all capitalized words / acronyms - Changing all the digits to 8, except when it is 1 3. Constructing a bigram library of ~ 50k 42

43 Examples of normal unique bigrams in the bigram library 43

44 Example of Report checker output by bigram approach Than mature: The endometrial cavity mass identified: A well-formed mass is not identified 44

45 Checking new gross texts for bigrams not in the library Two weeks of gross description texts (1717 cases) % of bigrams not in the library, flagging 3% of the sentences 2. 10% of the flagged sentences contain errors, 1.9% of the cases 45

46 Examples of the errors Confusing errors where from margin / away from margin uterine process / uterine corpus port material / soft material Very minor errors is an blue /is a blue is it is 0.3 x 0.2 cm / is a 0.3 x 0.2 cm 46

47 Checker learning: user updating of the bigram library Select the entries with no mistake (darker color) and then click the button below to teach the program 47

48 Bigram method vs other methods Other methods: searching for needles in a hay stack (you need to know what needles look like). Bigram method: identifying stuff that is not hay (you only need to know what hay looks like catching errors not seen before). 48

49 Implications More errors than we think: bigram / sex Universally implementable: database: information system-independent High benefit-cost ratio : very little increased effort Substantive effects on patient care: Avoid conveying wrong information Enable PA and pathologists to focus on the professional component of the tasks 49

50 Questions Jay J. Ye, MD, PhD, Pathologist Dahl-Chase Pathology Associates (207) I greatly appreciate the staff in the gross room in their dedication and effort to reduce the numbers of errors in the preliminary reports Please complete the online session evaluation 50