Detection and Classification of Calcifications on Digital Breast Tomosynthesis and 2D Digital Mammography: A Comparison

Transcription

1 Women s Imaging Original Research Spangler et al. Digital Breast Tomosynthesis Versus 2D Digital Mammography Women s Imaging Original Research FOCUS ON: M. Lee Spangler 1 Margarita L. Zuley 2 Jules H. Sumkin 2 Gordan Abrams 2 Marie A. Ganott 2 Christiane Hakim 2 Ronald Perrin 2 Denise M. Chough 2 Ratan Shah 2 David Gur 3 Spangler ML, Zuley ML, Sumkin JH, et al. Keywords: calcifications, digital breast tomosynthesis, digital mammography DOI: /AJR Received March 29, 2010; accepted after revision July 7, This work was supported by a research grant from Hologic. 1 Department of Radiology, University of Pittsburgh Medical Center, 3950 Presby South Towers, 200 Lothrop St., Pittsburgh, PA Address correspondence to M. L. Spangler (spanglerml@upmc.edu). 2 Department of Radiology, Magee-Women s Hospital, University of Pittsburgh, Pittsburgh, PA. 3 Department of Radiology, Radiology Imaging Research, University of Pittsburgh, Pittsburgh, PA. AJR 2011; 196: X/11/ JOURNAL American Roentgen Ray Society CLUB WOMEN S IMAGING Detection and Classification of Calcifications on Digital Breast Tomosynthesis and 2D Digital Mammography: A Comparison OBJECTIVE. The purpose of this article is to compare the ability of digital breast tomosynthesis and full field digital mammography (FFDM) to detect and characterize calcifications. MATERIALS AND METHODS. One hundred paired examinations were performed utilizing FFDM and digital breast tomosynthesis. Twenty biopsy-proven cancers, 40 biopsyproven benign calcifications, and 40 randomly selected negative screening studies were retrospectively reviewed by five radiologists in a crossed multireader multimodal observer performance study. Data collected included the presence of calcifications and forced BI-RADS scores. Receiver operator curve analysis using BI-RADS was performed. RESULTS. Overall calcification detection sensitivity was higher for FFDM (0.84% [95% CI, %]) than for digital breast tomosynthesis (0.75% [95% CI, %]). In the cancer cohort, 75 (76%) of 99 interpretations identified calcification in both modes. Of those, a BI-RADS score less than or equal to 2 was rendered in three (4%) and nine (12%) cases with FFDM and digital breast tomosynthesis, respectively. In the benign cohort, 123 (62%) of 200 interpretations identified calcifications in both modes. Of those, a BI-RADS score greater than or equal to 3 was assigned in 105 (85%) and 93 (76%) cases with FFDM and digital breast tomosynthesis, respectively. There was no significant difference in the nonparametric computed area under the receiver operating characteristic curves (AUC) using the BI-RADS scores (FFDM, AUC = 0.76 and SD = 0.03; digital breast tomosynthesis, AUC = 0.72 and SD = 0.04 [p = ]). CONCLUSION. In this small data set, FFDM appears to be slightly more sensitive than digital breast tomosynthesis for the detection of calcification. However, diagnostic performance as measured by area under the curve using BI-RADS was not significantly different. With improvements in processing algorithms and display, digital breast tomosynthesis could potentially be improved for this purpose. C alcifications are a frequent finding in mammography and can have a variety of causes. Specific patterns of breast calcifications have been associated with an increased positive predictive value of malignancy, whereas others portend benign pathologic abnormalities [1]. Full field digital mammography (FFDM) depicts calcifications extremely well, in large part because of high contrast resolution and processing algorithms that have been tailored to reveal calcifications. Several studies comparing digital mammography to screen film mammography have shown that digital mammography is superior in cancer detection in part because of superior detection of ductal carcinoma in situ [2 4]. However, to date, the overall positive predictive value is essentially the same in most studies [2, 3, 5]. Digital breast tomosynthesis has been shown to reduce recall rates by resolving overlapping structures seen on mammography [6], but there is some concern that it may not depict calcifications as well as traditional mammography [7]. Several studies have examined the clinical performance of digital breast tomosynthesis compared with that of FFDM [7 9], but, to our knowledge, no studies have specifically addressed the detection and characterization of calcifications. In addition, questions remain about the optimal display method of tomosynthesis for calcium evaluation. We present results from a multimode multireader retrospective study of 100 paired examinations and five readers to begin exploring these questions. 320 AJR:196, February 2011

2 Digital Breast Tomosynthesis Versus 2D Digital Mammography Materials and Methods Case Selection Digital breast tomosynthesis examinations used in this study were acquired under an institutional review board approved protocol for other research purposes that included a signed informed consent by the participant. Eligible subjects who participated in our institutional review board approved protocol for acquiring digital breast tomosynthesis examinations included, among others, women who presented for routine annual screening mammography and women who were found to have calcifications on FFDM that warranted biopsy. For the group of women enrolled because of suspicious calcifications, the tomosynthesis study was performed on the day of and before the biopsy. All FFDM examinations were obtained as part of routine clinical care on our system (Selenia, Hologic). All tomosynthesis procedures were performed on a research system. All FFDM and digital breast tomosynthesis studies included bilateral craniocaudal (CC) and mediolateral oblique (MLO) views. The tomosynthesis system acquires 15 low-dose projection images through a 30 arc in approximately 4.3 seconds per view and uses roughly the same dose per view as a standard mammogram. After acquisition, the data from the projection images are used to reconstruct parallel 1-mm-thick slices, the number of which varies with the thickness of the compressed breast. Examinations used for this study included all subjects who were enrolled in our protocol because of calcifications that were undergoing biopsy and that had the most current digital breast tomosynthesis acquisition parameters (4.3-second acquisition time). This included 20 biopsy-proven malignancies and 40 biopsy-proven benign cases. In addition, 40 screening cases categorized as BI-RADS 1 were randomly selected from our research database by a study coordinator. The studies were performed between November 17, 2008, and March 30, TABLE 1: Sensitivity of Calcification Detection by Reader and by Mode Reader Full Field Digital Mammography (%) Digital Breast Tomosynthesis (%) (50 75) 0.60 (47 72) (71 91) 0.68 (55 79) (81 97) 0.85 (73 92) (77 95) 0.78 (65 87) (81 97) 0.85 (73 92) All readers combined 84 (79 88) 75 (70 80) Note Data in parentheses are 95% CI. TABLE 2: Percentage of Calcification Accurately Detected by Each Mode and by Both Modes Calcification Type Full Field Digital Mammography Digital Breast Tomosynthesis Both Modes Malignant 90/100 (90) 79/99 (80) 75/99 (76) Benign 161/200 (81) 136/200 (68) 123/200 (62) Note Data are no. of calcifications/total no. of examinations (%). Radiologist Participants Five board-certified Mammography Quality Standards Act qualified radiologists with 7 36 years of experience participated in a fully crossed mode-balanced retrospective observer performance study. All readers had participated in prior reader studies of tomosynthesis and had experience viewing and interpreting tomosynthesis examinations [6, 10, 11]. All reading sessions were performed on commercially available mammography diagnostic workstations that have been developed by the manufacturer to display both tomosynthesis and FFDM images. The workstations have the ability to slab images to any thickness up to the compressed breast thickness. A standard hanging protocol was used to display both the FFDM and digital breast tomosynthesis images. CC and MLO standard views were available for interpretation in each mode. Additional diagnostic views, if any, that were obtained clinically and prior images were not included. Readers were taught the functionality of the workstation before reading and were allowed to slab the data as desired. Data Collection and Case Ratings Examinations were randomly divided into two groups of 50 cases each. Each group was then subdivided into two sessions, each of which contained only one mode. For the first group, three readers read the FFDM session first and two readers read the digital breast tomosynthesis images first; this was reversed for the second group to create a fully crossed study by readers and modes. Reading of the two modes was separated by a minimum of 4 weeks in each group. During interpretation, readers were asked to indicate whether they detected calcification clusters and to provide a forced BI- RADS score (1 5). Data Analysis Calcification detection truth for the purpose of sensitivity estimation was determined by the presence or absence of calcification on the conventional FFDM examinations. BI-RADS scores were evaluated with a multireader multicase receiver operator curve (ROC) analysis of variance (DBM MRMC version 2.2, Medical Image Perception Laboratory at University of Iowa). Curve fitting followed nonparametric trapezoidal method, and areas under the ROCs (AUCs) were computed for each reader in each mode. A p value of less than 0.05 (two sided) was considered statistically significant. Results One reader failed to submit data for a single patient with malignancy in digital breast tomosynthesis mode. In two additional instances, BI-RADS scores were not rendered for a patient with benign calcifications in digital breast tomosynthesis mode, despite the radiologist indicating the presence of calcification. Because five radiologists participated in this reader study, we ascertained a total of 500 interpretations under the FFDM mode and 499 interpretations under the digital breast tomosynthesis mode. Calcification Detection Comparison To assess for diagnostic differences between the two modes (if any), sensitivity, specificity, and matched sample analyses were performed. Overall calcification detection sensitivity was higher for FFDM (84% [95% CI, 79 88%]) than for digital breast tomosynthesis (75% [95% CI, 70 80%]). Sensitivity of calcification detection was also higher for the FFDM mode than for the digital breast tomosynthesis mode for each of the readers (Table 1). Specificity was also higher for FFDM (71% [95% CI, 64 77%]) than for digital breast tomosynthesis (64% [95% CI, 56 70%]). Calcification detection for both modes was assessed separately for the cohorts with biopsy-proven malignant and benign histopathologic abnormalities. Malignant calcifications were detected in 90 (90%) of 100 FFDM interpretations and in 79 (80%) of 99 digital breast tomosynthesis interpretations (Table 2). Malignant calcification was identified in both modes in 75 (76%) of 99 interpretations. Benign calcification was detected in 161 AJR:196, February

3 Spangler et al. (81%) of 200 interpretations with FFDM and in 136 (68%) of 200 interpretations with digital breast tomosynthesis. Benign calcification was identified correctly in both modes in 123 (62%) of 200 interpretations. Of the calcifications accurately detected using FFDM, 14 cancers and 38 benign calcifications were not detected with digital breast tomosynthesis. Of the calcifications accurately detected using digital breast tomosynthesis, four cancers and 13 benign calcifications were not detected with FFDM. Calcification Characterization Calcification characterization was evaluated by analyzing BI-RADS scores in cases in which calcification was detected in both modes. When malignant calcifications were detected in both modes, a BI-RADS score less than or equal to 2 was rendered in nine (12%) of 74 cases with digital breast tomosynthesis and in three (4%) of 75 cases with FFDM. When benign calcifications were accurately detected in both modes, a BI-RADS score greater than or equal to 3 was assigned in 93 (77%) of 121 cases with digital breast tomosynthesis and in 105 (85%) of 123 cases with FFDM. BI-RADS scores were not recorded in the digital breast tomosynthesis mode for one malignant and two benign calcifications. Table 3 shows the distribution of cases by BI-RADS score and by mode for cases in which calcifications were identified in both modes. ROC Analysis Table 4 shows the results of the multireader multicase AUC analysis of variance using BI-RADS scores. Overall, the AUC was greater for FFDM (AUC = 0.76; SD = 0.03) than for digital breast tomosynthesis (AUC = 0.72; SD = 0.04). The AUC was greater with FFDM than with digital breast tomosynthesis for each of the readers. However, the mean difference between the two modes was not significantly different (p = ). The performance curves of pooled data for all readers are shown in Figure 1. Discussion Identifying and characterizing calcifications are challenging tasks during mammography interpretation. It is a perceptual challenge for the radiologist, and it is a technical challenge for the vendors to balance the acquisition and display parameters to achieve optimally displayed images. Digital mammography has proven to reveal calcifications TABLE 3: BI-RADS Distribution of Cases in Which Calcification Was Detected in Both Digital Breast Tomosynthesis and Full Field Digital Mammography Modes BI-RADS Score TABLE 4: Area Under the Receiver Operator Curve Values Between Modes for BI-RADS Scores Reader Malignant Calcification Digital Breast Tomosynthesis well because of the high contrast resolution that this technology provides. Furthermore, computer-aided detection (CAD) and diagnosis devices have been developed to aid in calcification perception, classification, and characterization, and they perform quite well for this purpose [12, 13]. In order for breast tomosynthesis to satisfactorily compete with FFDM in clinical care, the technology will have to approximate FFDM in its ability to perform in both detection and classification of these lesions. In this study, we sought to compare FFDM (without CAD) and digital breast tomosynthesis with regard to the detection and characterization of breast calcifications, both to understand how radiologists would perform the perceptual tasks of identification and classification and also to learn how close the technology has come with respect to the technical aspects of acquisition and display. Our data suggest that, using FFDM, radiologists more sensitively detect calcifications than when using digital breast tomosynthesis. These results are in general agreement with those found in other studies [7]. At this Full Field Digital Mammography Full Field Digital Mammography Mode Digital Breast Tomosynthesis Mean a Note Data are area under the receiver operator curve values. a p = (not significant). Digital Breast Tomosynthesis Benign Calcification Full Field Digital Mammography Total 74 a a 123 Note Data are no. of cases. a Although calcification was detected, BI-RADS scores were not provided in one malignant and two benign cases, all in digital breast tomosynthesis mode. point in the evolution of digital breast tomosynthesis, this finding is not surprising given the perceptual challenges of interpreting tomosynthesis. First, radiologists are well versed in the standard display of digital mammography and have well-developed search patterns. Although our radiologists TPF FPF Fig. 1 Performance curves of pooled data for all readers. Data are nonparametric receiver operator curves for full field digital mammography (squares) versus digital breast tomosynthesis (diamonds). FPF = false-positive fraction, TPF = true-positive fraction. 322 AJR:196, February 2011

4 Digital Breast Tomosynthesis Versus 2D Digital Mammography have had prior experience interpreting tomosynthesis during prior research projects performed in our group, digital breast tomosynthesis is still not used to make patient care decisions and is a relatively new and unfamiliar technology. As a result, search patterns for digital breast tomosynthesis are not well established, which could result in oversight of a subtle cluster of calcium. Second, tomosynthesis images are reviewed as individual slices or as small user-defined bundled groups of slices, or slabs. Therefore, calcification conspicuity becomes dependent on user-defined slice thickness and location (Figs. 2A and 2B). Although increasing the slice thickness will increase the ability to perceive a 3D configuration of a cluster of calcium in the breast, the spatial resolution of each individual calcification is compromised with slabbing. Therefore, there are tradeoffs with each type of display, and the optimization of these display parameters is not yet known. Ultimately, optimal slabbing may be cluster dependent, and we recognize that, in this respect, allowing readers to define slab thickness could potentially confound our results. Third, compared with digital mammography, there are many more images to view with digital breast tomosynthesis, which can, in turn, lead to reader fatigue. Such a phenomenon could be overcome with increasing use of and exposure to the technique, as well as A Fig. 2 Full field digital mammogram of right breast. A, Craniocaudal projection image reveals fine pleomorphic calcifications (arrows) in linear ductal distribution. Mediolateral oblique projections confirmed them in upper outer quadrant. B, In 1-mm-thick tomographic slice, calcification is indistinct compared with full field digital mammography. Two of five readers identified calcification in right upper outer quadrant with full field digital mammography but not with tomosynthesis. Biopsy returned ductal carcinoma in situ. B developing display algorithms and CAD that aid in the detection task. Several technical factors could also contribute to the increased sensitivity of FFDM. First, acquisition time could contribute to motion artifact and thereby less distinct depiction of calcification. Because the acquisition time of tomosynthesis is longer (4.3 seconds in the data set used in this study), there may be motion artifact that could result in obscuring small calcifications. Second, because tomosynthesis is still a research tool, the exact exposure parameters (including number of low-dose projection images, dose per image, peak kilovoltage, and milliamperes), processing algorithms, dynamic range display, and tools for workstation functionality are being continually evaluated and upgraded. Further refinements in image acquisition and display will occur as radiologists gain experience and give feedback to vendors for improvements. The results of this preliminary study are reasonably encouraging in that the difference in diagnostic performance of the two techniques was not statistically significant. However, we recognize that the trend in favor A B Fig. 3 Full field digital mammogram of left breast. A, Mediolateral oblique projection image reveals cluster of fine pleomorphic calcifications (arrow) in upper breast barely projecting through dense glandular tissue. Craniocaudal projections confirmed them in upper outer quadrant. B, Calcifications (arrow) on 10-mm-thick tomogram are better appreciated because of decrease in overlapping breast tissue. Biopsy result was fibrocystic change with associated calcification. AJR:196, February

5 Spangler et al. FOR YOUR INFORMATION of FFDM was notable in that the AUC was greater for the FFDM mode for each of the five readers in this relatively small data set (only 100 examinations). Nevertheless, we think that our findings may at least partially reflect a combination of perceptual hurdles as well as suboptimal digital breast tomosynthesis acquisition and visualization protocols. Tomosynthesis can decrease the difficulty of interpreting overlapping breast tissue (Figs. 3A and 3B), and studies have supported the diagnostic performance of digital breast tomosynthesis in conjunction with [6] or independently of FFDM [9, 14]. There are several limitations to our study. First, our study is biased in favor of the FFDM mode because all cases with positive findings were initially identified as depicting calcification by FFDM, resulting in the recall recommendation before the tomosynthesis acquisition. By definition, all biopsy-proven calcium clusters were visible by FFDM. Second, though we are able to determine that, with this data set, our readers were better able to identify calcium with FFDM, we did not collect any data regarding image quality from the readers to understand exactly why FFDM did better than digital breast tomosynthesis. Whether the different performance was due to differences in lesion conspicuity, lesion distinctness, reader fatigue, or relative lack of familiarity with digital breast tomosynthesis was not ascertained. Third, our study design allowed radiologists to review bilateral two-view breast examinations in an attempt to mimic the clinical screening setting. We did not assess whether the index finding noted on screening FFDM was in the same breast on digital breast tomosynthesis. Moreover, we did not examine whether the lesion detected on digital breast tomosynthesis was a new finding. This limitation in study design could confound any possible differences between the two modes. Finally, the study involved only 100 examinations and five observers, with FFDM being read without CAD, resulting in relatively large 95% CIs, thereby limiting our ability to assess small differences between modes. We recognize that BI-RADS scores were more often overestimated for benign calcifications in both modes. This finding is likely the result, in large part, of selection bias because all cases with calcifications were recalled for biopsy after screening FFDM. We also note that the type of ROC analysis performed here namely, nonparametric (or trapezoidal) estimates of AUC values underestimates performance levels in absolute terms in most instances and at times underestimates it substantially. Therefore, it is the relative difference between modes (i.e., the difference in AUC values) rather than the absolute performance values in terms of AUC values themselves that is of primary interest in this article. In conclusion, FFDM was both more sensitive and specific than digital breast tomosynthesis for the detection of calcium clusters in this preliminary study, although the results were not statistically significant under the study conditions. However, the results are encouraging in that the difference between the technologies is relatively small in absolute terms and could possibly be overcome with improvements in acquisition and display of tomosynthesis. Acknowledgments We thank Linda Lovy for assistance in data collection and database organization. References 1. Burnside ES, Ochsne JE, Fowler KJ, et al. Use of microcalcification descriptors in BI-RADS 4th edition to stratify risk of malignancy. Radiology 2007; 242: Del Turco MR, Mantelli P, Ciatto S, et al. Fullfield digital versus screen-film mammography: comparative accuracy in concurrent screening cohorts. AJR 2007; 189: Hambly NM, McNicholas MM, Phelan N, et al. Comparison of digital mammography and screenfilm mammography in breast cancer screening: a review in the Irish Breast Screening Program. AJR 2009; 193: Pisano ED, Gatsonis C, Hendrik E, et al. Diagnostic performance of digital versus film mammography for breast-cancer screening. N Engl J Med 2005; 353: Yamada T, Saito M, Ishibashi T, et al. Comparison of screen-film and full-field digital mammography in Japanese population-based screening. Radiat Med 2004; 22: Gur D, Abrams GS, Chough DM, et al. Digital breast tomosynthesis: observer performance study. AJR 2009; 193: Poplack SP, Tosteson TD, Kogel CA, Nagy HM. Digital breast tomosynthesis: initial experience in 98 women with abnormal digital screening mammography. AJR 2007; 189: Gennaro G, Toledano A, di Maggio C, et al. Digital breast tomosynthesis versus digital mammography: a clinical performance study. Eur Radiol 2010; 20: Kontos D, Bakic PR, Carton AK, et al. Parenchymal texture analysis in digital breast tomosynthesis for breast cancer risk estimation: a preliminary study. Acad Radiol 2009; 16: Good WF, Abrams GS, Catullo VJ, et al. Digital breast tomosynthesis: a pilot observer study. AJR 2008; 190: Zuley ML, Bandos AI, Abrams GS, et al. Time to diagnosis and performance levels during repeat interpretations of digital breast tomosynthesis: preliminary observations. Acad Radiol 2010; 17: Birdwell RL. The preponderance of evidence supports computer-aided detection for screening mammography. Radiology 2009; 253: Elter M, Horsch A. CADx of mammographic masses and clustered microcalcifications: a review. Med Phys 2009; 36: Teertstra HJ, Loo CE, van den Bosch MA, et al. Breast tomosynthesis in clinical practice: initial results. Eur Radiol 2010; 20:16 24 The reader s attention is directed to the study guide pertaining to this Journal Club article, which can be accessed online at the article link labelled Study Guide. For more information on Journal Clubs, see Evidence-Based Radiology A Primer in Reading Scientific Articles in the July 2010 AJR at AJR:196, February 2011