Computer-Aided Detection in Screening CT for

Yuan et al. Screening CT for Pulmonary Chest Imaging Original Research C M E D E N T U R I C L I M G I N G JR 2006; 186:1280 1287 0361 803X/06/1865 1280 merican Roentgen Ray Society Y O Ren Yuan 1 Patrick M. Vos 2 Peter L. Cooperberg 2 Yuan R, Vos PM, Cooperberg PL F Keywords: computer-aided detection, CT, low-dose CT, lung cancer, pulmonary nodules DOI:10.2214/JR.04.1969 Received December 27, 2004; accepted after revision March 14, 2005. 1 Department of Radiology, Vancouver General Hospital, Vancouver, British Columbia, V5Z 1M9 Canada. 2 Department of Radiology, St. Paul s Hospital, 1081 Burrard St., Vancouver, British Columbia, V6Z 1Y6 Canada. ddress correspondence to P. L. Cooperberg. Computer-ided Detection in Screening CT for Pulmonary OBJECTIVE. Our objective was to evaluate the performance of a computer-aided detection (CD) system for pulmonary nodule detection using low-dose screening CT images. MTERILS ND METHODS. One hundred fifty consecutive low-dose screening CT examinations were independently evaluated by a radiologist and a CD pulmonary nodule detection system (R2 Technology) designed to identify nodules larger than 4 mm in maximum long-axis diameter. ll discrepancies between the two techniques were reviewed by one of another two radiologists working in consensus with the initial interpreting radiologist, and a true nodule count was determined. Detected nodules were classified by size, density, and location. The performance of the initial radiologist and the CD system were compared. RESULTS. The radiologist detected 518 nodules and the CD system, 934 nodules. Of the 1,106 separate nodules detected using the two techniques, 628 were classified as true nodules on consensus review. Of the true nodules present, the radiologist detected 518 (82%) of 628 nodules and the CD, 456 (73%) of 628 nodules. ll 518 radiologist-detected nodules were true nodules, and 456 (49%) of 934 of CD-detected nodules were true nodules. The radiologist missed 110 true nodules that were only detected by CD. In six patients, these were the only nodules detected in the examination, changing the imaging follow-up protocol. CD identified 478 lesions that on consensus review were false-positive nodules, a rate of 3.19 (478/150) per patient. CONCLUSION. CD detected 72.6% of true nodules and detected nodules in six (4%) patients not identified by radiologists, changing the imaging follow-up protocol of these subjects. In this study, the combined review of low-dose CT scans by both the radiologist and CD was necessary to identify all nodules. ung cancer is the leading cause L of cancer-related death [1, 2], with the current fatality rate exceeding that of the next three most common cancers (breast, prostate, and colorectal) combined. Multiple studies have shown that low-dose screening helical CT scans can detect peripheral lung cancers at an early stage [2 4]. Because of the prevalence of benign, stable lung nodules in current and former smokers, once a nodule is detected, serial follow-up scans must be performed to detect growth. The introduction of MDCT scanners has led to lung cancer screening studies with a larger number of thinner slices, resulting in the detection of more nodules [5, 6]. This increase in the number of images per CT examination makes the process of CT interpretation more time consuming and tedious for the radiologist. This can lead to decreased detection sensitivity for nodules caused by reviewer fatigue [1, 7 9]. Therefore, computerized methods for nodule detection to assist the radiologist may become important. Recently, computer-aided detection (CD) systems for lung nodules have been developed. Work has already begun to determine if CD can serve as a stand-alone replacement for the radiologist screener, or, if not, can it serve a role as a second pair of eyes to supplement the radiologist screener [1, 7, 8, 10 12]. We evaluated a CD system (ImageChecker CT LN-1000, R2 Technology) at our institution. The purpose of this study was to evaluate the value of this CD system for automated pulmonary nodule detection and potential as a stand-alone or as a supplement to the radiologist. Materials and Methods Image Data Between pril 2003 and February 2004, 150 consecutive patients (85 men, 65 women; mean 1280 JR:186, May 2006

Screening CT for Pulmonary Fig. 1 Computer-aided detection workstation user interface. Transaxial section at middle lung zone level contains lesion highlighted by green circle (right image panel). Yellow line indicates anatomic level of displayed section in coronal maximum intensity projection of pulmonary vascular tree (left upper image panel). 3D volume rendering (left lower image panel) of lesion displays anatomic location of lesion and relationship between lesion and surrounding structures (e.g., vessels and pleura). age, 61 ± 13 [SD] years) had low-dose unenhanced CT examinations for lung cancer screening or follow-up of previously detected pulmonary nodules. CT studies were performed on an MDCT scanner (LightSpeed, GE Healthcare) with a single breath-hold of less than 20 sec. cquisition parameters were identical in all patients (high speed mode; 1.5:1 pitch; table speed, 15 mm/sec; 1.25 mm collimation, 120 kvp, 80 ms). The original axial images with a 1.25-mm slice were used for the CD system. Images were reconstructed with 2.5-mm slice thickness for radiologist evaluation. These images were displayed on a conventional Windows-based (Microsoft) workstation with commercially available viewing software (efilm, Merge Technologies) and viewed by a radiologist. The average number of slices per patient at 1.25 mm was 404 (range, 194 525 slices). utomated Nodule Detection Method The CD system consists of two parts: a CD server and a CD workstation. CT data are transferred from the CT scanner to the CD server over the network using the DICOM protocol. The CD server accepts and analyzes the scans by segmenting the lung parenchyma from the vessels, mediastinum, and chest wall. Other bridging techniques are used to include lesions that may be touching the chest wall. The CD workstation is user controlled. The findings of CD are presented in three windows on the monitor: the original 2D axial images; a lung nodule map (an anteroposterior projection such as a chest X-ray), and a 3D rendering image (Fig. 1). Suspected lesions are circled in green on the nodule map. When a green-circled nodule is clicked with the mouse, the appropriate axial slice will appear, with the suspected nodule encircled in green. lso, the appropriate region on the 3D window appears with the suspected nodule colored green. The 3D image can be rotated and otherwise modified for the radiologist to decide if it is a true nodule or not. The size, volume, and density of the nodule are displayed on the left. nodule can be added by the radiologist and it will be surrounded by a green hexagon. The size, volume, and density are then also determined by the CD system. Radiologist and CD Performance First, all CT examinations were interpreted by a radiology fellow experienced in detection of pulmonary nodules, using a PCS workstation (efilm, Merge Technologies) with a 2.5-mm slice thickness. Then the CT was processed by the CD system. The results of the two techniques were compared, and a final decision was made. If there was a discrepancy between the radiologist and the CD system, consensus was made with another of the two radiologists. The suspected nodules detected by CD were divided into four groups: both-positive (BP) referred to the true nodules detected by both CD and the radiologist, true-positive (TP) referred to the true nodules detected by CD but missed by the radiologist, false-negative (FN) referred to the true nodules detected by the radiologist but missed by CD, and false-positive (FP) referred to the structures detected by CD as a nodule but rejected by the radiologists. The location of the true nodules was classified as follows [7]. subpleural nodule had pleural contact. peripheral nodule was within 2 cm of, but not touching, the pleura. hilar nodule was within 2 cm of the hilum. central nodule was situated between the peripheral and hilar zones. The nodules were separated into the following three groups by diameter: less than 4 mm, greater than or equal to 4 mm but smaller than or equal to 10 mm, and greater than 10 mm. We also classified them into solid pulmonary nodules and nonsolid pulmonary nodules using the peak H of 100 as described by Miller D et al. (presented at the 2003 annual meeting of the Radiological Society of North merica). The performance of the CD system was evaluated in terms of nodule detected (especially additional nodules detected) and the number of false-positives per CT study. The reasons for CD false-negatives and false-positives were analyzed. Results There were 1,106 suspected nodules detected by either CD or radiologists. fter radiologist review, 628 were finally scored as true nodules. Of these, 518 (83%) of 628 nodules were detected by the radiologist and 456 (73%) of 628 were detected by CD. Of those, there were 110 (17.5%) true nodules (Fig. 2), which would have been missed by the radiologist without using the CD system. Importantly, six of the 150 (4%) patients were initially considered as normal by the radiologist and wouldn t have been recommended for follow-up without CD. However, CD missed 172 of 628 true nodules. In 56 (37.3%) of 150 patients, the results of nodule detection (number and location) corresponded between a radiologist and a CD system; of these, 16 patients did not have any nodules. Finally, 478 (51.2%) of 934 of the nodules detected by CD in 122 patients were rejected by the radiologist(s) as false-positives, for a rate of 3.19 (range, 0 26) falsepositives per CT study. JR:186, May 2006 1281

Yuan et al. n overview of the size distribution of true nodules and the corresponding detection performance of both techniques is given in Table 1. Of the 115 small nodules (< 4 mm) missed by CD (Table 1), besides the size algorithm limitation, lower density and contact to normal structures (e.g., pleura and vessel) further decreased the detection performance. Thirtyseven nodules had a peak attenuation value of Fig. 2 Transverse CT images with target field of view show solid nodules (green circles) only detected by computer-aided detection (CD) system. C, Small (< 4 mm) central nodule (), moderate (4 mm diameter 10 mm) peripheral nodule (B), and moderate (4 mm diameter 10 mm) nodule at hilum (C) were surrounded by similar-sized vessels, which make them difficult for radiologist to distinguish on 2D image. B C 1282 JR:186, May 2006

Screening CT for Pulmonary TBLE 1: Size Distribution and Nodule Detection Performance of Both Techniques Found by Computer-ided Detection and Radiologist(s) Diameter Total (n) Computer-ided Detection Only < 4 mm 291 62 114 115 4 mm diameter 10 mm 310 48 212 50 > 10 mm 27 0 20 7 Total 628 110 346 172 Fig. 3 Small solid nodules (< 4 mm; CT peak > 100 H) were missed by computeraided detection because of size algorithm. Some were in contact with the pleura () or vessels (B). Radiologist(s) Only B less than 100 H; 34 were in contact with the pleura (Fig. 3) or vessel (Fig. 3B). Fifty moderate nodules (4 mm diameter 10 mm) were missed by CD. Of these, six subpleural nodules were missed because they had pleural contact (Fig. 4). mong 44 of 50 central and peripheral nodules without obvious pleural contact, 21 had lower density (CT peak < 100 H) (Fig. 4B); 11 nodules were attached to a linear pleural tag or the normal intrapulmonary structures, such as fissure (Fig. 4C) and vessel (Fig. 4D) and thereby were excluded by the segmentation algorithm. There was no explanation for the other 12 nodules that were missed (Fig. 4E). Seven large nodules (> 10 mm) were missed by CD because of their continuity with normal structures, such as pleura (Fig. 5), fissure, or vessels. The location of 628 true nodules and the detection performances of both techniques are presented in Table 2. We found the detection performance of the CD system and the radiologist to be complementary (Fig. 6); the CD sensitivity was higher in hilar (100%) and the central area (84%), and the radiologist s sensitivity was higher in the peripheral area (86%) and subpleural area (98%). Of the 22 central nodules missed by CD, six were less than 4 mm. Two of the six were attached to normal structures (Fig. 3B); the other four had low density (CT peak < 100 H), which made them more difficult for CD to identify. With the other 16 central nodules greater than 4 mm, CD failed to detect them, possibly because of their lower density (n =8; CT peak < 100 H), or because they were abutting normal structures (n =6) and were thereby excluded by the segmentation algorithm. No explanation was determined for the remaining two nodules. Thirty-four subpleural nodules were missed by CD presumably because of the segmentation algorithm. In addition, 27 of them were smaller than 4 mm (Fig. 3). There were 116 peripheral nodules missed by CD; 82 (71%) of 116 were smaller than 4 mm (moreover, 31 of 82 had a peak attenuation value < 100 H). mong the remaining 34 (29%) nodules greater than 4 mm, a CT peak of less than 100 H was seen in 13 nodules (Fig. 4B). ttachment to the pleura (Fig. 5), vessel (Fig. 4D), and fissure (Fig. 4C) was seen in another 13 nodules; no obvious reason was found in the remaining eight nodules (Fig. 4E). CT peak greater than or equal to 100 H was seen in 561 (89.3%) of 628 true pulmonary nodules, which is considered to repre- JR:186, May 2006 1283

Yuan et al. sent a solid pulmonary nodule (Miller D et al., presented at the 2003 annual meeting of the Radiological Society of North merica). The detection sensitivity of solid pulmonary nodules was 80% (448 of 561 nodules) with CD and 81% for the radiologist (455 of 561 nodules). These are very close. Solid nodules were found only by CD and missed by the radiologist in 106 (18.9%) of 561 nodules (Fig. 2). CD missed 113 (20.1%) of 561 solid nodules. Seventy-eight (70%) of those were less than 4 mm (Fig. 3). mong the remaining 35 (30%) of 113 nodules greater than or equal to 4 mm, attachment to the pleura (Figs. 4 and 5) or fissure (Fig. 4C) and vessels (Fig. 4D) was seen in 23 of 35; no obvious reason was found in the remaining 12 (Fig. 4E). Reasons for the 478 false-positives from CD and thus defined as false-positive (FP) include vessel, 262 (55%); pleural, 114 (24%); B C Fig. 4 Computer-aided detection missed moderate (4 mm diameter 10 mm) nodules (surrounded by green hexagons) that resulted from contact with pleura (), lower density (CT peak < 100 H) (B), or contact with normal intrapulmonary structures (e.g., major fissure [C] and vessel [D]). Some were missed inexplicably (E). (Fig. 4 continues on next page) D 1284 JR:186, May 2006

Screening CT for Pulmonary Fig. 4 (continued) Computer-aided detection missed moderate (4 mm diameter 10 mm) nodules (surrounded by green hexagons) that resulted from contact with pleura (), lower density (CT peak < 100 H) (B), or contact with normal intrapulmonary structures (e.g., major fissure [C] and vessel [D]). Some were missed inexplicably (E). TBLE 2: Nodule Location Distribution and Detection Performance of Both Techniques Found by Computer-ided Location n a Computer-ided Detection Only (%) Detection and Radiologist(s) (%) Radiologist(s) Only (%) Hilar nodule 14 8/14 (57.1) 6/14 (42.9) 0/14 (0) Central nodule 141 45/141 (32.0) 74/141 (52.4) 22/141 (15.6) Peripheral nodule 386 55/386 (14.2) 215/386 (55.7) 116/386 (30.1) Subpleural nodule 87 2/87 (2.3) 51/87 (58.7) 34/87 (39) a Total n = 628. scars, 57 (12%); and others (e.g., bone, ground glass, and chest wall soft tissue), 45 (9%) (Fig. 7). Discussion CD systems hold promise for helping radiologists to increase detection sensitivity. For detecting pulmonary nodules, sensitivity is increased by performing thinner slices with MDCT [8]. Our study shows that compared with a radiologist interpreting a reasonable number of medium-slice thickness (2.5-mm) images, there can be an increased detection of pulmonary nodules in the range of 21.2% by the addition of CD using thinner slices (1.25 mm). wide variation exists in the detection sensitivity by CD of pulmonary nodules in previously published studies [1, 7, 8, 10]. The detection sensitivity was 84% in a study by rmato et al. [1], who used CD retrospectively in 31 patients with missed lung cancer. In Brown et al. [8], CD detected 74% of nodules in 15 patients who had lung cancer. In a study by Wormanns et al. [7], only 38% of nodules were detected in 85 healthy subjects. Goo et al. [10] studied 50 volunteers and found a sensitivity of 65%. In our study, CD had a 73% sensitivity in 150 patients. False-positive rates range from three to 13 per CT study [1, 7, 8, 10]. In our study, there were 3.19 false-positives per study. In our study, there was a high prevalence of nodules (mean, 4.18 per patient). This is because most cases were pulmonary nodule follow-ups of the original screening examinations performed in outside facilities, so very few cases had no nodules. Only 16 of the total 150 patients had no nodules detected, which might be a selection bias. Despite that, there E were six patients whose nodules were only detected by CD. In a screening situation, these patients would have been lost to followup, possibly missing cancers. The high false-negative rate of CD limits its application as a stand-alone technique. Missing true nodules by CD in our study was predominately due to size limitation ( 4 mm), attenuation limitation (CT peak 100 H), and the segmentation algorithm (CD only recognizes nodules entirely surrounded by lung parenchyma). Our results showed that these three factors are interactive in influencing the CD detection performance. However, this CD system still picked up 62 of the total 291 small nodules (< 4 mm) (Table 1). The reason is that although the current CD algorithm targets nodules with diameters greater than 4 mm and less than 30 mm (in which a nodule is more typically described as a mass), the CD algorithm also examines the nodular findings with diameters between 2 and 4 mm. For these smaller nodular findings, a stricter set of criteria on shape (i.e., more spherical) and location (i.e., clearly not part of adjacent structures) are applied to determine whether they are presented as CD findings. Further development of the technology hopefully will overcome these deficits. Our results also showed that CD and the radiologist worked in a complementary fashion in different lung zones because neither of them was able to find every nodule (Fig. 6). The radiologist has little difficulty in finding the peripheral and subpleural nodules even if they are small because there are no vessels of similar size near the pleural surface (Figs. 3 and 4). CD is more sensitive in showing central nodules (Fig. 2), especially hilar nodules among the large vessels, which are prone to be misinterpreted as vessels and overlooked by the radiologist. The high false-positive rate of CD requires the radiologists to look at each suspected nodule to confirm its authenticity. In our study, the false-positive nodules were seen in 122 of the total 150 patients, which contribute to 3.19 false-positive rate per study. lthough the analysis of nodules using one-way system (either by a CD system or a radiologist) was satisfactory in 56 (37.3%) patients of our study, radiologists still needed to look at 46 of 56 CT scans to reject the falsepositive. The causes of false-positives are vessel (54%), pleura (24%), and scar (12%). The others (10%) include consolidation, bone structure, and soft tissue of the chest wall. This has been described in other studies [1, 7, JR:186, May 2006 1285

Yuan et al. 8, 10, 12]. The particular software of our CD system facilitates differentiating falsepositives by the 3D-rendering image. The capability to rotate this image facilitates the distinction of a true nodule from the pulmonary vascular tree or pleural thickening. lthough the false-positive rate is a drawback, CD should have a high sensitivity even at the expense of a low specificity. Fig. 5 Computer-aided detection (CD) failed to detect large nodules (> 10 mm), presumably due to segmentation algorithm. contacted pleura (in circle) and normal intrapulmonary structures and therefore were extracted by CD segmentation algorithm. 120% 90% 60% 30% 0% 100% Hilar 43% 84% Central 68% 70% 86% Peripheral CD Radiologist 61% 98% Subpleural Fig. 6 Detection sensitivity between computer-aided detection (CD) and radiologist in nodules at different locations. Fig. 7 Transverse CT section shows computer-aided detection false-positive detections (in circle), which were vessels () or the pleura (B). B 1286 JR:186, May 2006

Screening CT for Pulmonary The main characteristic to diagnose malignant nodules is their growth over time [13 18]. lgorithms are already being developed to do temporal comparisons on followup studies. ll previously detected nodules would be automatically assessed to see if they have increased in volume and at what rate. This feature probably will greatly enhance the diagnostic value of CD systems in CT screening for early lung cancer [19]. CD software is useful to supplement radiologists detection performance. However, at present, it is not adequate as a stand-alone procedure. Furthermore, all suspected lesions detected by CD must be interpreted by radiologists to rule out false-positives. In the future, temporal comparison should further improve the usefulness of CD in the early detection of lung cancer. cknowledgment We wish to express appreciation to John Mayo for help in reviewing the manuscript. References 1. rmato SG 3rd, Li F, Giger ML, MacMahon H, Sone S, Doi K. Lung cancer: performance of automated lung nodule detection applied to cancers missed in a CT screening program. Radiology 2002; 225:685 692 2. Nawa T, Nakagawa T, Kusano S, Kawasaki Y, Sugawara Y, Nakata H. Lung cancer screening using low-dose spiral CT: results of baseline and 1-year follow-up studies. Chest 2002; 122:15 20 3. Henschke CI, McCauley DI, Yankelevitz DF, et al. Early Lung Cancer ction Project: overall design and findings from baseline screening. Lancet 1999; 354:99 105 4. Diederich S, Wormanns D, Semik M, et al. Screening for early lung cancer with low-dose spiral CT: prevalence in 817 asymptomatic smokers. Radiology 2002; 222:773 781 5. Fischbach F, Knollmann F, Griesshaber V, Freund T, kkol E, Felix R. Detection of pulmonary nodules by multislice computed tomography: improved detection rate with reduced slice thickness. Eur Radiol 2003; 13:2378 2383 6. Diederich S, Semik M, Lentschig MG, et al. Helical CT of pulmonary nodules in patients with extrathoracic malignancy: CT-surgical correlation. JR 1999; 172:353 360 7. Wormanns D, Fiebich M, Saidi M, Diederich S, Heindel W. utomatic detection of pulmonary nodules at spiral CT: clinical application of a computer-aided diagnosis system. Eur Radiol 2002; 12:1052 1057 8. Brown MS, Goldin JG, Suh RD, McNitt-Gray MF, Sayre JW, berle DR. Lung micronodules: automated method for detection at thin-section CT: initial experience. Radiology 2003; 226:256 262 9. Li F, Sone S, be H, MacMahon H, rmato SG 3rd, Doi K. Lung cancers missed at low-dose helical CT screening in a general population: comparison of clinical, histopathologic, and imaging findings. Radiology 2002; 225:673 683 10. Goo JM, Lee JW, Lee HJ, Kim S, Kim JH, Im JG. utomated lung nodule detection at low-dose CT: preliminary experience. Korean J Radiol 2003; 4:211 216 11. rmato SG 3rd, Giger ML, Moran CJ, Blackburn JT, Doi K, MacMahon H. Computerized detection of pulmonary nodules on CT scans. RadioGraphics 1999; 19:1303 1311 12. wai K, Murao K, Ozawa, et al. Pulmonary nodules at chest CT: effect of computer-aided diagnosis on radiologists performance. Radiology 2004; 230:347 352 13. Shaham D, Guralnik L. The solitary pulmonary nodule: radiologic considerations. Semin Ultrasound CT MR 2000; 21:97 115 14. Yankelevitz DF, Reeves P, Kostis WJ, Zhao B, Henschke CI. Small pulmonary nodules: volumetrically determined growth rates based on CT evaluation. Radiology 2000; 217:251 256 15. Swensen SJ. Functional CT: lung nodule evaluation. RadioGraphics 2000; 20:1178 1181 16. Yankelevitz DF, Gupta R, Zhao B, Henschke CI. Small pulmonary nodules: evaluation with repeat CT preliminary experience. Radiology 1999; 212:561 566 17. Hasegawa M, Sone S, Takashima S, et al. Growth rate of small lung cancers detected on mass CT screening. Br J Radiol 2000; 73:1252 1259 18. Wang JC, Sone S, Feng L, et al. Rapidly growing small peripheral lung cancers detected by screening CT: correlation between radiological appearance and pathological features. Br J Radiol 2000; 73:930 937 19. Ko JP, Betke M. Chest CT: automated nodule detection and assessment of change over time preliminary experience. Radiology 2001; 218:267 273 JR:186, May 2006 1287