Identification of Missed Pulmonary Nodules on Low Dose CT Lung Cancer Screening Studies Using an Automatic Detection System

Transcription

1 Identification of Missed Pulmonary Nodules on Low Dose CT Lung Cancer Screening Studies Using an Automatic Detection System Carol L. Novak *a, Li Fan a, Jianzhong Qian a, Guo-Qing Wei a, David P. Naidich b a Siemens Corporate Research; 755 College Road East, Princeton, NJ b New York University Medical Center, st Avenue, New York, NY ABSTRACT Multi-slice CT (MSCT) scanners allow nodules as small as 3mm to be identified during screening. However the associated large data sets make it challenging for radiologists to identify all small nodules in a reasonable amount of time. Computer-aided detection may play a critical role in identifying missed nodules. 13 MSCT screening studies, initially interpreted as non-actionable by a radiologist, were selected from participants in a lung cancer screening study. The study protocol defines actionable studies as those containing at least 1 solid non-calcified nodule larger than 3mm, for which follow-up studies are recommended to exclude interval growth. An automatic detection algorithm was applied to the 13 studies to determine whether it might detect missed nodules, and whether any of these were of sufficient size to be considered actionable. There were a total of 138 automatically detected candidate nodules, an average of 10.6 per patient. 83 candidates were characterized as true positives, yielding a positive predictive value of 60.1%. 10 automatically detected candidates were judged to be actionable nodules greater than 3mm in diameter. 6 of 13 (46%) patients had at least one actionable finding detected by the computer that had been overlooked in the initial exam. Keywords: CT lung cancer screening, automatic detection, computer-aided diagnosis 1. INTRODUCTION Lung cancer is the leading cause of cancer-related mortality in the United States and worldwide. The average five-year survival rate for all stages of lung cancer is only about 15% 1. However lung cancer diagnosed in the early stages has a much better prognosis, with reported five-year survival rates for stage I lung cancer of 60-70% 2-3. Therefore detection of lung cancer in the earliest stages may hold the key to improving cure rates. Low dose Computed Tomography (CT) imaging for screening of lung cancer is becoming increasingly popular, since it allows earlier detection of lung tumors than chest x-rays 4. Several studies are currently under way to determine whether CT screening will ultimately decrease disease specific mortality in patients with non-small cell lung cancer. Multi-slice CT (MSCT) scanners are capable of scanning the entire volume of the lungs in a single breath-hold, utilizing a slice collimation of 1mm or less, and at a low enough radiation dose to be acceptable for screening. With such high-resolution data, it is possible for radiologists to detect and examine very small, potentially malignant nodules. However, MSCT results in typical data sets of 300 to 600 images per patient. This presents a substantial clinical burden to examining physicians. Due to workflow constraints, many radiologists do not examine images reconstructed every 1mm, but instead look at a smaller set of images reconstructed at thicker intervals, such as 5mm thick images reconstructed every 4mm, or 7mm thick images reconstructed every 6mm. This reduces the number of images to examine to a more manageable 50 to 75 per study. Naturally, thicker sections make it more difficult for the reader to find small nodules, especially those less than 5mm in diameter. However even with the use of 1mm images, it is still quite possible for readers to miss nodules in the 3-5mm size range, principally due to obscuring structures such as vessels and airways. * carol.novak@scr.siemens.com Medical Imaging 2003: Image Perception, Observer Performance, and Technology Assessment, Dev P. Chakraborty, Elizabeth A. Krupinski, Editors, Proceedings of SPIE Vol (2003) 2003 SPIE /03/$

2 Due to the complexity of examination of MSCT data sets, some type of computerized assistance would be an extremely valuable asset to radiologists. Many research groups are currently developing and testing Computer Aided Diagnosis (CADx) systems for lung cancer screening with CT One of the goals of CADx is for the computer to automatically identify nodules that would otherwise have been missed by the examining physician. In this paper we present some preliminary data about the potential clinical impact of an automatic nodule detection system upon routine screening. 2. METHODS AND MATERIALS Thirteen low dose (40 mas) MSCT screening studies, initially interpreted as normal or non-actionable, were selected from participants evaluated at New York University Medical Center as part of the Early Detection Research Network (EDRN) study of lung cancer screening. The EDRN protocol calls for CT studies to be performed using 1 mm collimation, but with initial radiologic evaluation performed on 7mm reconstructed images, consistent with standard clinical care. According to the EDRN protocol, all solid nodules less than 3mm in diameter are interpreted as non-actionable. By contrast, actionable nodules are defined as solid non-calcified nodule greater than 3mm in diameter. Non-solid or pure ground glass nodules are considered actionable if they are greater than 5mm in diameter. Nodules with a homogeneous pattern of calcification are considered non-actionable regardless of size. An actionable nodule leads to the recommendation of some follow-up evaluation for the patient, typically a repeat MSCT scan in 3 months to exclude interval growth. In the case of very suspicious lesions, an immediate biopsy may be recommended rather than a repeat scan. Non-actionable nodules receive no follow-up other than a repeat yearly scan. These categories are shown in table 1. Non-actionable Solid nodules < 3mm in diameter Non-solid nodules < 5mm in diameter Calcified nodules Actionable Solid nodules > 3mm in diameter Non-solid nodules > 5mm in diameter Table 1: Description of actionable and non-actionable nodules according to the EDRN study protocol. Actionable nodules result in some recommendation for follow-up evaluation, typically a repeat scan in 3 months. A patient study is deemed non-actionable if it contained no actionable nodules detected during the radiologist screening on 7mm sections. An automatic detection algorithm developed at Siemens Corporate Research was applied to 1.25mm reconstructions from the non-actionable studies, to determine whether the algorithm might detect missed nodules, and whether any of these were of sufficient size to fall into the actionable category. The automatic detection system employed here is a knowledge-based automatic detection system with multiple modules developed for different nodule types 15. The system makes use of specialized algorithms to detect solitary nodules, nodules attached to vessels, and nodules attached to the chest wall. The modules are integrated to share processing information efficiently, to speed up processing and increase reliability. In addition the system contains specialized modules for removing false positives. The modular system is extensible to handle the inclusion of new detection systems, such as one for detecting sub-solid or ground glass nodules. The results of the automatic detection algorithm were validated by a thoracic radiologist experienced in interpreting low dose CT screening studies. For each computer-generated finding the radiologist determined whether the finding truly constituted a nodule, whether it was calcified, and whether it was above or below the size criteria established in the study protocol (see table 1). The sizes were initially determined by the placing a 3mm diameter circle on the axial slice where the nodule appeared largest, to determine whether the nodule fit completely within the circle. If the nodule extended outside the circle, it was considered to have diameter larger than 3mm in the axial plane, and thus actionable. 440 Proc. of SPIE Vol. 5034

3 The nodule sizes were also evaluated by a computer program that performs interactive segmentation. The segmentation program determines the greatest nodule diameter in any dimension, rather than the greatest diameter in the axial dimension 16. The greatest diameter in any direction will generally be slightly larger than the greatest axial diameter, and in no case smaller. As semi-automated or automated segmentation becomes increasingly available in the clinical workflow, radiologists will probably come to rely increasingly on measurements made in this way. For this paper, we report nodule sizes made both by appearance in the axial plane, and by computer measurement in all axes. Figure 1 shows one of the missed actionable nodules detected by the computer. Figure 1(b) shows a 3mm circle superimposed on the image, indicating that the nodule is larger than 3mm in diameter in the axial plane. The computer measured this nodule as 5.2mm in the largest diameter. (b) (a) Figure 1: Example of a missed actionable nodule detected by the computer. (a) shows the entire slice with a box around the detected nodule. (b) shows a magnification of the region inside the box in 1(a); the dark circle is exactly 3mm in diameter, indicating that the largest diameter of the nodule in this slice exceeds 3mm. (c) shows a shaded surface display of the segmented nodule in light gray. The darker gray structures are nearby vessels. (c) Figure 2 shows one of the missed non-actionable nodules detected by the computer. The nodule was judged to be nonactionable because it fit completely inside the 3mm circle, indicating that it was smaller than 3mm axially. However the computer measured the diameter as 4.1mm in the largest dimension. The radiologist also retrospectively examined the 7 mm images containing the missed nodules to estimate whether they could in principle have been seen during the initial examination of thick sections. Nodules that were visible on the thick sections, generally those that were peripherally located, were judged retrospectively visible. Nodules that could not be seen on the 7mm images, generally those obscured by vessels and centrally located, were judged to be retrospectively not visible. Proc. of SPIE Vol

4 (b) (a) Figure 2: Example of a missed non-actionable nodule detected by the computer. (a) shows the entire slice with a box around the detected nodule. (b) shows the boxed region in more detail; the circle is exactly 3mm in diameter, indicating that the largest diameter of the nodule in this slice is less than 3mm. (c) shows a shaded surface display of the segmented nodule in light gray. The darker gray structures are nearby vessels, the chest wall, and some artifacts due to noise. Figure 3 shows a computer detected actionable nodule that was judged to have been retrospectively visible on the 7mm thick sections. 3(a) shows the appearance on 1mm thick slices and 3(b) shows the appearance on 7mm thick slices. The nodule is located fairly close to the periphery of the lungs and is not obscured by surrounding vessels. (c) (a) Figure 3: Example of an actionable nodule that was judged retrospectively visible on thick sections. The nodule is centered within the box. (a) shows the appearance on 1mm sections and (b) shows the appearance on 7mm thick sections. (b) 442 Proc. of SPIE Vol. 5034

5 Figure 4 shows a computer detected actionable nodule that was judged not to have been retrospectively visible on the thick sections. The nodule is located centrally and thus not distinguishable from vessels of similar size in the vicinity. (a) Figure 4: Example of an actionable nodule that was not retrospectively visible on thick sections. (a) shows the nodule on 1mm sections and (b) shows the nodule on 7mm thick sections. (b) 3. RESULTS There were a total of 138 automatically detected candidate nodules in the 13 patients, an average of 10.6 per patient. The number of detections per patient ranged from 1 to 53. Each candidate nodule was classified into one of the following categories: false positive, an actionable non-calcified nodule with size above 3mm, a non-actionable noncalcified nodule with size below 3mm, a calcified nodule of any size, or a non-nodule abnormality such as scarring or atelectasis. Since the automatic detection system is not currently designed to detect pure ground glass nodules, there is no separate category for their detection. The categories are shown in table 2. Category Description 0 False positive 1 Actionable nodule 2 Non-calcified nodule < 3mm in diameter 3 Calcified nodule of any size 4 Non-nodule abnormality Table 2: Categories for classifying automatically detected nodule candidates. 55 of the automatically detected candidates were classified as false positives in category 0, an average of 4.2 per patient, with a range of 0 to 17 false positives per patient. The remaining 83 candidates were categorized as belonging to one of the four categories of true positives, yielding an average of 6.4 per patient, with a range of 0 to 47 per patient. 47 (34.1%) of the 138 candidates were calcified nodules in category 3, and 15 (10.9%) were interpreted as non-nodule abnormalities in category of the 138 candidates (15.2%) were interpreted as non-calcified nodules. Size alone was used to assign them to the actionable category 1 or non-actionable category 2. Using the apparent diameter on an axial slice to determine size, 10 of the nodules were judged large enough to be actionable, and 11 were judged below 3mm in diameter and thus nonactionable. These results are summarized in table 3. Proc. of SPIE Vol

6 Number of % of Per patient Category findings total Average Minimum Maximum 0 false positive % actionable % non-actionable % calcified % other abnormality % total % Table 3: Results of automatic detection by category However when measuring the greatest diameter in any dimension, all 21 non-calcified nodules were at least 3mm in diameter. This is not a coincidence; the fact is that the automatic detection system has been configured to only report nodules that are at least 3mm in their largest diameter. It is likely that the system could have detected additional nodules smaller than 3mm in diameter, but the algorithm is currently designed to ignore smaller findings. If clinical practice guidelines were to change to specify the detection of smaller nodules, of course it would be a simple matter to change the program to report smaller findings. These results are summarized in table 4. Since the current clinical practice is to judge actionability by the appearance of nodule size on axial slices, we will follow this guideline to classify only 10 of the 21 non-calcified nodules as actionable under the current clinical standard. Number of detected noncalcified nodules with given axial diameter Number of detected noncalcified nodules with given diameter in any direction > 3mm < 3mm 11 0 Table 4: Categorization of nodule sizes by maximum axial diameter, or by maximum diameter in any direction The 10 clearly actionable nodules that had been missed by the examining physician and detected by the computer were divided among 6 patients. In other words, 6 of the 13 patients, or 46%, had at least one actionable finding. If instead we consider all 21 non-calcified nodules that were at least 3mm in diameter in some axis, the 21 findings were divided among 10 patients (77%). Only 3 of the 13 patients (23%) had no computer-detected non-calcified nodules. These findings are shown in the graph in figure 5. The EDRN protocol for patients enrolled in the study specifies that non-calcified nodules above 3mm in diameter are actionable. However other studies may use different size thresholds. 62% of patients had at least one missed nodule larger than 4mm in diameter when measured in any direction. 31% of patients had at least one missed nodule larger than 5mm. In this sample of 13 patients, the computer did not find any missed non-calcified nodules larger than 8mm in diameter. Figure 6 shows the percentage of patients containing at least 1 missed nodule at various size thresholds. In retrospect, 28 (41.2%) of 68 true positive nodules in categories 1, 2 and 3 could be identified on the initial 7 mm images once their location was known. This includes 20 of 47 (42.6%) calcified nodules. 6 of 10 (60%) nodules larger than 3mm in diameter on axial slices could be retrospectively identified on the thick sections. By contrast, only 2 of 11 (18.2%) nodules with axial diameter <3mm could be retrospectively seen on thick sections. The rest of the nodules were not visible on corresponding thick sections, due to their central location or proximity to an adjacent vessel. These results are summarized in table Proc. of SPIE Vol. 5034

7 Percentage of patients with at least 1 actionable nodules missed by reader and detected by computer 23% Patients with >= 1 actionable nodule 46% Patients with >= 1 possibly actionable nodule Patients with 0 noncalcified nodules 31% Figure 5: Automatic detection of nodules not seen by examining radiologist, according to percentage of patients containing at least 1 finding. Definitely actionable nodules were >3 mm in diameter on an axial slice. Possibly actionable nodules were >3 mm in diameter in some other dimension, but less than 3mm axially. 100% Patients with at least one non-calcified nodule of given size % of patients 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 77% 62% 31% 23% 8% 0% >3 >4 >5 >6 >7 >8 largest diameter (mm) in any direction Figure 6: Percentage of patients where the computer detected at least one missed non-calcified nodule greater than various thresholds. Proc. of SPIE Vol

8 1 2 3 Nodule Retrospective examination on thick sections Category Visible Not visible Total Actionable nodule Nonactionable nodule Calcified nodule 6 (60.0%) 4 (40.0%) 10 2 (18.2%) 9 (81.8%) (42.6%) 27 (57.4%) 47 Table 5: Results of retrospective examination of computer-detected nodules, to determine whether they could have been seen during reader examination of thick sections. 5. DISCUSSION Studies of lung cancer screening with low-dose CT are finding that one-quarter to one-half of patients have an actionable finding at the first screening Our results suggest that with the use of automatic nodule detection software, this percentage could rise substantially. In this investigation we found that almost half of the screening patients who were deemed by the screening examiner to have no actionable nodules, actually had a nodule that if seen might have led to a recommendation for follow-up. If we expand the criteria from 3mm in diameter in the axial direction to 3mm in any direction, still more patients were found to have nodules large enough to warrant action. We have observed that fairly often nodules that are just over 3mm in their greatest diameter will have a cross-section slightly below 3mm in the axial plane. As software for automatically or semi-automatically measuring nodules becomes more widely available, radiologists may come to rely upon it to determine nodule sizes and thus actionability. We predict that changing the measurement criteria from 3mm in the axial plane to 3mm in any direction would inevitably lead to an increase in actionable nodules if the threshold for action were kept constant. Our automatic detection system is not yet able to detect all nodules found by physicians. In particular non-solid or pure ground glass nodules are not currently detected by the system. Although the computer did detect several nodules missed by the examining physicians, it should not be concluded that the computer is more skilled at detection than a radiologist, but rather that it is complementary. The lungs have a very complex structure of vessels and airways, and it is impractical for a reader to examine every millimeter in detail. We argue that this should be the job of the computer. 6. CONCLUSIONS In this investigation, we show that an automatic nodule detection system is frequently able to find nodules not reported by the examining physician, but which if seen might have led to the recommendation for further action. The computer found at least 1 non-calcified nodule larger than 3mm in axial diameter in 6 out of 13 or 46% of non-actionable studies. The computer program also detected non-calcified nodules that were less than 3mm in the axial diameter, although larger than 3mm in some other axis, in an additional 4 patients (31%). Although automatic lung nodule detection is still in the stages of development and testing, we conclude that the potential is clearly there to aid physicians in the detection of small, potentially malignant nodules that would have otherwise been overlooked during routine screening examinations. REFERENCES 1. J. H. Schiller, Current standards of care in small-cell and non-small-cell lung cancer, Oncology 61 Supp l, pp. 3-13, M. T. van Rens, A. B. de la Riviere, H. R. Elbers, J. M. van Den Bosch, Prognostic assessment of 2,361 patients who underwent pulmonary resection for non-small cell lung cancer, I, II, and IIIA, Chest 117(2), pp , Proc. of SPIE Vol. 5034

9 3. R. Shah, S. Sabanathan, J. Richardson, A.J. Mearns, C. Goulden, Results of surgical treatment of stage I and II lung cancer, The Journal of Cardiovascular Surgery, 37, pp , C. I. Henschke, D. I. McCauley, D. F. Yankelevitz, D. P. Naidich, G. McGuinness, O. S. Miettinen, D. M. Libby, M. W. Pasmantier, J. Koizumi, N. K. Altorki, J. P. Smith, Early lung cancer action project: overall design and findings from baseline screening, The Lancet, 354, pp , S. G. Armato III, F. Li, M. L. Giger, H. MacMahon, S. Sone, K. Doi, Lung Cancer: Performance of Automated Lung Nodule Detection Applied to Cancers Missed in a CT Screening Program, Radiology, 225(3), pp , M. S. Brown, M. F. McNitt-Gray, J. G. Goldin, R. D. Suh, D. R. Aberle, Patient-specific models for lung nodule detection and surveillance in CT images, Proceedings of the SPIE Conference on Medical Imaging, 4322, pp , P. Croisille, M. Souto, M. Cova, S. Wood, Y. Afework, J.E. Kuhlman, and E.A. Zerhouni, Pulmonary nodules: improved detection with vascular segmentation and extraction with spiral CT, Thoracic Radiology, vol. 197, pp , M. Fiebich, D. Wormanns, W. Heindel, Improvement of method for computer-assisted detection of pulmonary nodules in CT of the chest, Proceedings of the SPIE Conference on Medical Imaging, 4322, pp , M. N. Gurcan, N. Petrick, B. Sahiner, H-P. Chan, P. N. Cascade, E. A. Kazerooni, L. M. Hadjiiski, Computerized lung nodule detection on thoracic CT images: combined rule-based and statistical classifier for false positive reduction, Proceedings of the SPIE Conference on Medical Imaging, 4322, pp , W. J. Kostis, A. P. Reeves, D. F. Yankelevitz, and C. I. Henschke, Three-dimensional segmentation of solitary pulmonary nodules from helical CT scans, Computer Assisted Radiology and Surgery, pp , M. Kubo, T. Yamamoto, Y. Kawata, N. Niki, K. Eguchi, H. Ohmatsu, R. Kakinuma, M. Kaneko, M. Kusumoto, N. Moriyama, K. Mori, H. Nishiyama, CAD system to support comparative reading for lung cancer based on helical CT images, Computer Assisted Radiology and Surgery, ICS 1230, pp , C. L. Novak, L. Fan, J. Qian, G. Kohl, D. P. Naidich, An interactive system for CT lung nodule identification and examination, Computer Assisted Radiology and Surgery, ICS 1230, pp , H. Takizawa, S. Yamamoto, T. Matsumoto, Y. Tateno, T. Iinuma, M. Matsumoto, Recognition of lung nodules from x-ray CT images using 3D MRF models, Computer Assisted Radiology and Surgery, pp , R. Wiemker, P. Rogalla, A. Zwartkruis, T. Blaffert, Computer aided lung nodule detection on high resolution CT data, Proceedings of the SPIE Conference on Medical Imaging, J. Qian, L. Fan, G. Wei, C. L. Novak, B. Odry, H. Shen, L. Zhang, D. P. Naidich, J. P. Ko, A. N. Rubinowitz, G. Kohl, Knowledge-based Automatic Detection of Multi-type Lung Nodules from Multi-detector CT Studies, Proceedings of the SPIE Conference on Medical Imaging, 4684, L. Fan, J. Qian, B. Odry, H. Shen, D. P. Naidich, G. Kohl, E. Klotz; Automatic Segmentation of Pulmonary Nodules by Using Dynamic 3D Cross-correlation for Interactive CAD Systems ; Medical Imaging 2002: Image Processing; M. Sonka and M. J. Fitzpatrick, editors; Proceedings of SPIE Volume 4684, pp , C.I. Henschke, D.F. Yankelevitz, R. Mirtcheva, G. McGuinness, D. McCauley, O.S. Miettinen, CT screening for lung cancer: frequency and significance of part-solid and nonsolid nodules ; AJR Am J Roentgenol 178(5), pp , S. Diederich, D. Wormanns, M. Semik, M. Thomas, H. Lenzen, N. Roos, W. Heindel, Screening for early lung cancer with low-dose spiral CT: prevalence in 817 asymptomatic smokers ; Radiology 222(3), pp , Proc. of SPIE Vol