A superior chest x-ray computer-aided detection (CXR CAD) application: a reader study

Transcription

1 A superior chest x-ray computer-aided detection (CXR CAD) application: a reader study Poster No.: C-0865 Congress: ECR 2011 Type: Scientific Paper Authors: M. T. Freedman 1, S.-C. B. Lo 1, S. Marshall 2, C. M. Bromley 2 ; 1 2 Washington, DC/US, Mt. Airy, MD/US Keywords: DOI: Lung, Computer applications, Respiratory system, Conventional radiography, Plain radiographic studies, CAD, Computer Applications-Detection, diagnosis, Screening, Perception image, Neoplasia /ecr2011/C-0865 Any information contained in this pdf file is automatically generated from digital material submitted to EPOS by third parties in the form of scientific presentations. References to any names, marks, products, or services of third parties or hypertext links to thirdparty sites or information are provided solely as a convenience to you and do not in any way constitute or imply ECR's endorsement, sponsorship or recommendation of the third party, information, product or service. ECR is not responsible for the content of these pages and does not make any representations regarding the content or accuracy of material in this file. As per copyright regulations, any unauthorised use of the material or parts thereof as well as commercial reproduction or multiple distribution by any traditional or electronically based reproduction/publication method ist strictly prohibited. You agree to defend, indemnify, and hold ECR harmless from and against any and all claims, damages, costs, and expenses, including attorneys' fees, arising from or related to your use of these pages. Please note: Links to movies, ppt slideshows and any other multimedia files are not available in the pdf version of presentations. Page 1 of 9

2 Purpose It is well recognized that radiologists miss small lung cancers on chest radiographs (references 1-2) Computer-aided detection (CADe) aids radiologists in the detection of lung nodules and lung cancer on chest radiographs (3-10) CADe is approved for marketing by the US Food and Drug Administration (FDA) and has the CE mark for the European Union A newer system, OnGuard 5.1, is compared to, and is found superior to, a prior version, OnGuard 1.0 (both Riverain Medical, Miamisburg, OH, USA) with both reader studies and machine tests Methods and Materials OnGuard 5.1 and OnGuard 1.0 (Riverain Medical, Miamisburg, OH, USA) are commercially available in the US and European Union. These CADe systems are designed to aid physicians detect small lung nodules including cancer on chest radiographs -OnGuard 5.1 was introduced in OnGuard 1.0 was introduced in 2001 OnGuard identifies solitary pulmonary nodules from 9 to 30 mm in size Population of readers: 15 US Board Certified Radiologists Cases: 81 primary lung cancer, 172 without cancer Modality: Posteroanterior chest radiographs Reader design: Page 2 of 9

3 Sequential reads each system 1+ month apart (7-10) Re-randomized for each read and each reader Statistical design -Reader Study: Localization Receiver Operating Characteristic (LROC) analysis (11-15) -Machine test: Dorfman, Berbaum, Metz (16) method for differences in Sensitivity and Specificity Outcome measures -Reader study Areas under the LROC curve (Az) Sensitivity, Specificity, False Positives per Image (FP/I) -Machine tests Cancer detection Sensitivity False Positives per Images Results Page 3 of 9

4 Fig. References: M. T. Freedman; Oncology, Lombardi Comprehensive Cancer Center, Washington, UNITED STATES OF AMERICA Page 4 of 9

5 Reader Study Location Receiver Operating Characteristic (LROC): The mean change in area under the LROC between readings made with the OnGuard 1.0 and OnGuard 5.1 CADe systems (DAUCv1-v5) was (95% CI: to ) (Statistically Significant p # 0.05). OnGuard 5.1 was superior to OnGuard 1.0 Differences in correct location: Sensitivity, Specificity, Positive (PPV) and Negative Predictive Value (NPV) Fig. References: M. T. Freedman; Oncology, Lombardi Comprehensive Cancer Center, Washington, UNITED STATES OF AMERICA Page 5 of 9

6 In this chart, when the upper 95% confidence limit is below the zero line, the result is significant at the p # 0.05 Thus sensitivity and NPV are significant Machine Test Sensitivity and False Positive (FP) rate for OnGuard 1.0 and OnGuard 5.1 Machine Sensitivity FP Rate (Standard Deviation) OnGuard (1.96) OnGuard ** 1.36 (1.31)** **Note: much higher machine sensitivity for cancer detection with OnGuard 5.1. Note: many fewer false positives per image with OnGuard 5.1. False Positive Rate Radiographs Mean Difference N p-value All 3.66 (3.42 to 3.91) 253 < Cancer-free 3.73 (3.44 to 4.03) 172 < Cancer 3.52 (3.05 to 3.99) 81 < Conclusion The chest radiograph is an effective method for detecting small lung cancers (17-18) Radiologists using US FDA approved commercial computer-aided detection (CADe) systems can increase their rate of detection of small nodules including small lung cancers (3-7) Page 6 of 9

7 This test compared two versions of a commercial CADe systems and showed that radiologists improved their detection of lung cancer with the newer version (OnGuard 5.1) Radiologists can detecty more small lung nodules, including more small nodules due to primary lung cancer, with computer-aided detection (CADe) systems than without them CADe technology has improved since its initial commercial introduction 10 years ago, with Greater Sensitivity Much lower rate of false positives per image References 1. Austin JH, Romney BM, Goldsmith LS. Missed bronchogenic carcinoma: radiographic findings in 27 patients with a potentially resectable lesion evident in retrospect. Radiology 1992; 182: Quekel LG, Kessels AG, Goei R, van Engelshoven JM. Miss rate of lung cancer on the chest radiograph in clinical practice. Chest Mar;115(3): Chest 1999; 115: Li F, Engelmann R, Metz CE, Doi K, MacMahon H. Lung cancers missed on chest radiographs: results obtained with a commercial computer-aided detection program. Radiology 2008; 246: Chen JJ, White CS. Use of CAD to evaluate lung cancer on chest radiography. J Thorac Imaging 2008; 23: White CS, Pugatch R, Koonce T, Rust SW, Dharaiya E. Lung nodule CAD software as a second reader: a multicenter study. Acad Radiol 2008; 15: White CS, Flukinger T, Jeudy J, Chen JJ. Use of a computer-aided detection system to detect missed lung cancer at chest radiography. Radiology 2009; 252: Kobayashi T, Xu XW, MacMahon H, Metz CE, Doi K. Effect of a computeraided diagnosis scheme on radiologists' performance in detection of lung nodules on radiographs. Radiology 1996; 199: Page 7 of 9

8 8. Freedman M. State-of-the-art screening for lung cancer (part 1): the chest radiograph. Thorac Surg Clin 2004; 14: Freedman MT, Osicka T. Computer aided diagnosis for decision support in thoracic imaging. In: Siegel E, Reiner B, Erickson B, eds. Decision Support in the Digital Medical Environment. Leesburg, VA: SCAR (Society for Computer Applications in Radiology), 2005; Riverain Medical Group. RapidScreen Summary of Safety and Effectiveness Swensson RG. Unified measurement of observer performance in detecting and localizing target objects on images. Med Phys 1996; 23: DeLuca PM. ICRU Report 79: Receiver Operating Characteristic Analysis in Medical Imaging. Journal of the ICRU 2008; Popescu LM. Nonparametric ROC and LROC analysis. Med Phys 2007; 34: Chakraborty DP, Yoon HJ. Operating characteristics predicted by models for diagnostic tasks involving lesion localization. Med Phys 2008; 35: SAS Institute Inc. SAS software (Version 9.1.3,.) ; 9.1.3: 16. Dorfman DD, Berbaum KS, Metz CE. Receiver operating characteristic rating analysis. Generalization to the population of readers and patients with the jackknife method. Invest Radiol 1992; 27: Oken MM, Marcus PM, Hu P, et al. Baseline chest radiograph for lung cancer detection in the randomized Prostate, Lung, Colorectal and Ovarian Cancer Screening Trial. J Natl Cancer Inst 2005; 97: Hocking WG, Hu P, Oken MM, Et al. Lung Cancer Screening in the Randomized Prostate, Lung, Colorectal, and Ovarian (PLCO) Cancer Screening Trial. J Natl Cancer Inst May 4. [Epub ahead of print] Personal Information M.T. Freedman, MD, MBA Department of Oncology Lombardi Comprehensive Cancer Center 3900 Reservoir Road, N.W. Page 8 of 9

9 Washington, DC , USA Page 9 of 9