Reactive Quality Audit Pilot and Results: Quality Assurance Strategies for Cancer Surveillance

Transcription

1 Reactive Quality Audit Pilot and Results: Quality Assurance Strategies for Cancer Surveillance Rebecca Ehrenkranz, MPH; Valentina Petkov, MD; Clara Lam, PhD; Amy Solis; Steve Friedman; Steven Cheng, PhD; David Dilts, PhD; Lynne Penberthy, MD; Serban Negoita, MD June 12, 2018

2 Methods and Descriptive Analysis Reactive trigger: tumor size in breast and pancreatic cancer Benchmarking 1: Completeness by year and registry Benchmarking 2: Biologic plausibility Benchmarking 3: Cross tabulations!2

3 Why tumor size in breast and pancreatic cancer? Tumor size is key for staging and treatment, however: No benchmarks exist for quality of tumor size data collection Given importance of this data item, we wanted to conduct this assessment to validate SEER data quality and/or create a corrective action plan Note 1: A pre-evaluation is a SEER*Stat data audit, and no external source triggered the quality audit What was considered a trigger for CS tumor size? High proportion of unknown/missing values Very large tumor size for anatomic structure Discrepancies between size and T element of TNM Why breast and pancreas? Understand data differences in cancers diagnosed via screening vs. no screening, and early stage vs. late stage!3

4 Benchmarking 1: Completeness by year and registry Site-specific considerations Pancreas: Completeness improves from ~60% of cases with a tumor size value in 2004 to ~80% of such cases by 2014 Hypothesis: Given natural history of pancreatic cancer, likelihood of metastasis at diagnosis, potential lack of surgery may not obtain distinct meaningful tumor size value Breast: Completeness improves from ~93% of cases with a tumor size value in 2004 to ~95% of such cases by 2014 Hypothesis: Screening and public awareness leads to earlier diagnosis!4

5 Completeness in Pancreas Tumor Size Values, by Registry % Pancreas TS, Proportion Complete (Excluding DCO/Autopsy) Upper Fence: 80.8% 75.00% 50.00% Lower Fence: 59.4% 25.00% 0.00% A B C D E F G H I J K L M N O P Q R Proportion complete 25 percentile 75 percentile IQR Lower Fence Upper Fence 67.4% 72.8% 5.4% 59.4% 80.8%!5

6 Completeness in Breast Tumor Size Values, by Registry 1.2 Breast TS, Proportion Complete (Excluding DCO/Autopsy) Upper Fence: 97.9% 0.9 Lower Fence: 80.8% A B C D E F G H I J K L M N O P Q R Proportion complete 25 percentile 75 percentile IQR Lower Fence Upper Fence 94.6% 95.9% 1.3% 92.7% 97.9%!6

7 Benchmarking 2: Biologic Plausibility Cutoffs are preliminary: to determine true benchmark, additional comprehensive assessment of DMS/registry data is necessary (especially for accuracy) Breast: Current plausibility cutoff: 200+ mm (rationale: literature, organ size) Pancreas: Current plausibility cutoff: 100+ mm (rationale: organ size)!7

8 Tumor Size in Pancreatic Cancer: Biologic Plausibility Pancreas TS AJCC Categories 1-20 mm tumors 8, mm tumors 39, mm tumors 33,090 Pancreas Tumor Size: Large Tumors mm tumors 31, mm tumors 1, mm tumors 870!8

9 Tumor Size in Breast Cancer: Biologic Plausibility Breast TS AJCC Categories 1-5 mm tumors 47, mm tumors 104, mm tumors 210, mm tumors 188, mm tumors 45,802 Breast Tumor Size: Large Tumors mm tumors 39, mm tumors 4, mm tumors mm tumors 550!9

10 Benchmarking 3: Cross tab of tumor size and AJCC Derived T Goal of cross tabs during QAP evaluation: Create/utilize/test means of categorizing and comparing data Assess tumor size in context of closely related variable Gain insight into data quality by looking for discrepancies Which results stand out as unusual based on what we already know and expect about this data item? Can we identify any unlikely results that may merit further exploration beyond initial QAP assessment?!10

11 Cross tabulations: Tumor size & AJCC Derived T Pancreas Tumor Size and AJCC 6 th Edition Derived T Cross Tabulation T1 T2 T3 T4 Row Total 1-20 mm 3, , , mm 0 16,085 22,645 9,449 48, mm 0 4,026 6,321 4,116 14, mm mm Breast AJCC 6th Edition Derived T Cross Tabulation 209 T1 T2 T3 T4 Row Total 1-20 mm 368, , , mm , , , mm ,479 10,637 44, mm Values highlighted in red are potential discrepancies!11

12 Conclusion Takeaway message Specific recommendations Lessons learned Questions!12

13 Tumor size Quality Audit conclusions Overall Correct relation between T and tumor size is a necessary condition for checking accuracy; but is not a sufficient condition Based on the low proportion of unknown tumor size, uniform distribution across registries, and the small proportion with discrepancies in cross tabs, no immediate action is recommended at this time Pancreas % Unknown: acceptable due to natural history of the cancer Biologic Plausibility: very few tumors outside of cutoff; suspected inaccuracy Cross tabs: 713 out of 71,115 tumor measurements had direct discrepancy Breast % Unknown: acceptable due to low proportion Biologic Plausibility: very few tumors outside of cutoff Cross tabs: 632 out of 603,459 tumor measurements had direct discrepancy!13

14 Lessons learned & recommendations Lessons learned from performing the Quality Audit: Need for detailed activity checklist Importance of consultations/expert opinion Considerations of the data item across time Individualized methodology Streamlining process Specific recommendations post Quality Audit: Edit Flags in SEER*Abs: in process Site-specific flags for large tumor sizes Example - automatic review of all 10+ cm/100+ mm tumors Create dropdown menu for units of measurement!14

15 Any questions or comments? Thank you!

16

17 Completeness in breast cancer by year and registry, !17

18 Completeness in pancreatic cancer by year and registry, !18