The Effect of Image Processing on the Detection of Cancers in Digital Mammography

Transcription

1 Medical Physics and Informatics Original Research Warren et al. Image Processing in Digital Mammography Medical Physics and Informatics Original Research Lucy M. Warren 1,2 Rosalind M. Given-Wilson 3 Matthew G. Wallis 4,5 Julie Cooke 6 Mark D. Halling-Brown 1 Alistair Mackenzie 1,2 Dev P. Chakraborty 7 Hilde Bosmans 8 David R. Dance 1,2 Kenneth C. Young 1,2 Warren LM, Given-Wilson RM, Wallis MG, et al. Keywords: digital mammography, image processing, observer performance DOI: /AJR Received August 27, 2013; accepted after revision December 3, This work is part of the OPTIMAM project and was supported by Cancer Research UK & Engineering and Physical Sciences Research Council Cancer Imaging Programme in Surrey in association with the Medical Research Council and Department of Health (England). D. P. Chakraborty was supported in part by grants from the U.S. Department of Health & Human Services, National Institutes of Health (ROI-EB and ROI-EB008688). 1 National Coordinating Centre for the Physics of Mammography, Royal Surrey County Hospital, NHS Foundation Trust, Guildford, GU2 7XX, UK. Address correspondence to L. M. Warren (lucy.warren@nhs.net). 2 Department of Physics, Faculty of Engineering and Physical Sciences, University of Surrey, Guildford, UK. 3 Department of Radiology, St. George s Healthcare NHS Trust, Tooting, London, UK. 4 Cambridge Breast Unit, Cambridge University Hospitals, NHS Foundation Trust, Cambridge, UK. 5 NIHR Cambridge Biomedical Research Centre, Cambridge, UK. 6 Jarvis Breast Screening and Diagnostic Centre, Guildford, UK. 7 Department of Radiology, University of Pittsburgh, Pittsburgh, PA. 8 University Hospitals Leuven, Leuven, Belgium. AJR 2014; 203: X/14/ American Roentgen Ray Society The Effect of Image Processing on the Detection of Cancers in Digital Mammography OBJECTIVE. The objective of our study was to investigate the effect of image processing on the detection of cancers in digital mammography images. MATERIALS AND METHODS. Two hundred seventy pairs of breast images (both breasts, one view) were collected from eight systems using Hologic amorphous selenium detectors: 80 image pairs showed breasts containing subtle malignant masses; 30 image pairs, biopsy-proven benign lesions; 80 image pairs, simulated calcification clusters; and 80 image pairs, no cancer (normal). The 270 image pairs were processed with three types of image processing: standard (full enhancement), low contrast (intermediate enhancement), and pseudo film-screen (no enhancement). Seven experienced observers inspected the images, locating and rating regions they suspected to be cancer for likelihood of malignancy. The results were analyzed using a jackknife-alternative free-response receiver operating characteristic (JAFROC) analysis. RESULTS. The detection of calcification clusters was significantly affected by the type of image processing: The JAFROC figure of merit (FOM) decreased from 0.65 with standard image processing to 0.63 with low-contrast image processing (p = 0.04) and from 0.65 with standard image processing to 0.61 with film-screen image processing (p = ). The detection of noncalcification cancers was not significantly different among the image-processing types investigated (p > 0.40). CONCLUSION. These results suggest that image processing has a significant impact on the detection of calcification clusters in digital mammography. For the three image-processing versions and the system investigated, standard image processing was optimal for the detection of calcification clusters. The effect on cancer detection should be considered when selecting the type of image processing in the future. I n digital mammography, the final stage in the image formation process is the application of image processing. Image processing is applied to enhance the diagnostic content within the image for example, by reducing the noise or enhancing the edges and image contrast. Each manufacturer applies its proprietary image-processing algorithm, and the image appearances resulting from this processing can be very different. A number of studies have been performed to investigate the effect of the difference in image appearance as a result of applying different image-processing algorithms: These studies include both preference studies [1, 2] and objective studies [3 9], and the results of these studies are mixed. Some of the objective studies found significant differences among different image-processing algo- rithms [3 5], and others did not find a significant difference [6 9]. However, these studies had the following limitations: Different imageprocessing algorithms were used on systems with different detectors [3], different radiologists inspected different images [8], localization was not included in the task [4], not all cases were from different patients to maintain the independence assumed in the analysis [6], the cancers tended to be obvious and easily detected in all image-processing types investigated [7], and only calcifications were investigated (i.e., noncalcification cancers were not studied) [5, 9]. Therefore, the effect of the difference in image appearance resulting from using different image-processing algorithms on the detection of the various radiologic features of breast cancer is still unclear. The purpose of this work was to perform a retrospective observer study to investigate AJR:203, August

2 Warren et al. the impact of different image-processing algorithms on the detection of cancers in digital mammography. Our approach overcame several of the limitations described previously. Subtle simulated calcifications and subtle noncalcification cancers were included in the study; all the observers inspected all images processed with all image-processing algorithms investigated; the cases were from different patients, thus maintaining independence of cases; the image-processing algorithms investigated were representative of the range currently used in clinical practice; and the study was based on objective measurements rather than observer preference. Materials and Methods The study protocol was approved by the regional research ethics committee. Our ethics committee did not require individual consent because our study used anonymous mammographic images and data. Case Collection An expert breast radiologist (25 years experience and 6000 screening mammography studies read annually) who did not take part in the study inspected the images and the associated clinical information of 238 patients with pathology-proven cancer. One hundred and eighty five of these women presented for screening and 53 were women showing symptoms of cancer. These cases were collected at two sites using eight mammography systems with amorphous selenium detectors (HologicSelenia and Hologic Dimensions, Hologic). In the United Kingdom National Health Service (NHS) Breast Screening Programme, women years old are invited to undergo mammographic screening every 3 years. Women older than 70 years may refer themselves. The examination consists of two views TABLE 1: Malignant Mammographic Abnormalities Used in This Study Abnormality No. (%) of Cases Soft-tissue density (mass) 40 (48.2) Architectural distortion 19 (22.9) Asymmetric density 24 (28.9) of each breast: a mediolateral oblique (MLO) view and a craniocaudal view. The radiologist marked the outline of each cancer in both views (MLO and craniocaudal) on digital images using an in-house web interface [10] and rated lesion conspicuity in each view (very subtle, subtle, or visible). Eighty cases with malignant noncalcification lesions (i.e., mass, architectural distortion, or asymmetric density) were selected from these 238 cases by the expert radiologist. The expert radiologist chose cases in which lesion conspicuity was subtle or very subtle but the lesions were still detectable within the images because obvious cancers would be easily detected under all image-processing options. In total, there were 83 noncalcification cancers in the 80 cases; one case was a patient who had bilateral cancer with noncalcification cancers in both breasts. A more detailed description of these cancers is given in Table 1. In addition, the expert radiologist also inspected the images of 52 consecutive patients with biopsy-proven benign lesions (50 screening-detected cases and two symptomatic cases) and marked the location of the lesions. The images of 30 patients with biopsy-proven benign lesions were randomly selected from these 52 cases for the study. These were a mixture of calcification and noncalcification lesions with a mixture of conspicuities. The images of 160 patients assessed as showing normal findings (i.e., patient not recalled from screening) were also used in the study. Because of the time constraints of the study, the patients had not yet returned for their next screening session, so these normal cases have not been followed up. The interval cancer rate has been shown to be 0.55 cancers per 1000 women screened for the 12-month period after negative findings on a screening study in the United Kingdom [11]. Therefore, of the 270 patients in our study, we could expect 0.15 interval cancers. Although ideally all patients with normal findings would have been followed up, we do not think this lack of follow-up has significantly impacted the results of this study. The number of subtle calcification clusters within the 238 cancer cases collected was smaller than the number of calcification clusters required for sufficient statistical power in this study. Therefore, instead simulated calcifications were inserted into 80 of the 160 images with normal findings. The method used for calcification cluster simulation is described in the next section. At the time of the study, we did not have simulation models of noncalcification cancers that were suitably advanced for use; therefore, images of real noncalcification cancers as described earlier were used. Calcification Cluster Simulation Simulated calcification clusters have been used in observer studies of this type in the past [5, 9, 12]. The main advantage of using simulated cancers is that it is very time-consuming to collect real subtle cancers because of the low prevalence of cancers in breast screening, whereas a database of simulated cancers can be generated in a shorter time period. An additional advantage of using simulated cancers for this study is that cancers detected in the clinic will be found by a radiologist viewing images acquired on a particular digital mammography A Fig. 1 For this study, images of calcification clusters were inserted into breast images that had originally showed normal findings. A C, Three examples of calcification clusters inserted into breast images. Regions shown are mm 2. B C 388 AJR:203, August 2014

3 Image Processing in Digital Mammography system with a particular image-processing algorithm applied. Therefore, if the clinically used image-processing algorithm is one of the arms in the study, because all the cancers have already been detected with this image-processing algorithm, there may be a bias in favor of this clinically used imageprocessing algorithm. When simulated cancers are used, this bias is removed. The simulated calcification cluster images were derived from unprocessed digital images of sliced mastectomy samples imaged on a digital microfocus specimen x-ray cabinet (model MX20 DC2, Faxitron X-Ray, FaxitronBioptics), at five times magnification with an effective pixel size of 10 μm. The clusters within the mastectomy samples were a mixture of ductal carcinoma in situ and invasive carcinoma. From these images, calcification clusters were extracted using a method described elsewhere [9, 13] to generate an image of the calcification cluster with only the surrounding background tissue removed. If inserted into the normal breast images without any other adjustments, the clusters within the image would appear different than if the cluster had been within the patient s breast when imaged. This difference results from differences in magnifications, increased scatter, and decreased subject contrast due to the increased thickness of surrounding breast tissue and the different imaging system and imaging parameters used when acquiring the cluster images and normal breast images. Therefore, before a simulated calcification cluster was inserted, its appearance was adjusted to account for these differences. This process has been described and validated elsewhere [14, 15]. Calcification clusters were inserted into the breast images in a range of locations while avoiding positions very close to the skin edge and nipple. The breasts into which the calcification clusters were inserted were randomly selected from all normal images collected, so the breasts were of all densities. When the location to insert a cluster was being selected, the breast tissue immediately surrounding the cluster was categorized as homogeneously fatty, a mixture of fatty and glandular tissue, or homogeneously glandular. The locations were selected so that there were equal numbers of clusters in the three broad categories of local breast density. In total, 89 calcification clusters were inserted with between one and three clusters inserted per case. Examples of simulated calcification clusters inserted into breast images are shown in Figure 1. It was important that the calcification clusters were realistic in appearance; therefore, once inserted within the breast images, the calcification clusters were inspected by the same expert radiologist who judged them to be realistic in appearance and location and to be subtle in appearance. Image Processing The 80 cases with inserted calcification clusters, 80 normal cases with no cancer present, 80 cases with noncalcification cancers, and 30 cases with biopsy-proven benign lesions made up the set of 270 cases used in the study. These 270 cases were processed using three different types of Hologic image processing (Fig. 2). The first type was the manufacturer s standard image processing and is referred to as the standard image processing (Fig. 2A). The second version has an intermediate amount of enhancement. This type is referred to as the low-contrast image processing (Fig. 2B). The third has no additional enhancement except skin edge enhancement, so it simulates the appearance of a film-screen image. This type is referred to as film-screen image processing (Fig. 2C). The lowcontrast and the film-screen image-processing algorithms were developed by Hologic for investigational purposes only. However, the three types of image processing were selected for this study to be representative of the wide range in image-processing al- A Fig year-old woman with noncalcification cancer. Expert radiologist classified lesion conspicuity in this case, which was used in study, as subtle. A C, Subtle noncalcification cancer is shown in insets with standard image processing (A), low-contrast image processing (B), film-screen image processing (C). B C AJR:203, August

4 Warren et al. gorithms seen on the different digital mammography systems currently used in clinical practice. For all the cases used in the study, images of both breasts were displayed but only one view (MLO or craniocaudal) was shown. Ideally, two views of both breasts would be displayed, as in the clinic. However, only 2D simulated calcification clusters were available and these could not be reoriented for simulation into both views. For normal and benign cases, the view was selected randomly (subject to the constraint that the benign lesions be visible in the randomly selected view); for cases containing noncalcification cancers, the view in which cancer was most subtle while remaining detectable was selected. Of the 270 image pairs used in the study, 142 were MLO views and 128 were craniocaudal views. Observer Study Protocol Seven mammography readers (six radiologists and one radiographer) who had 2 22 years of experience reading mammograms (2 7 years with digital mammography) and who read at least 5000 screening examinations per year took part in the observer study. All observers were certified to interpret mammography images as defined by the United Kingdom NHS Breast Screening Programme [16] and had experience reading similar images (i.e., from this manufacturer) in their clinical practice. Each observer inspected the images in six sessions, and a break was taken halfway through each session. The images were organized so that there was at least 2 weeks between each observer seeing the same image pair with different image processing applied, and the sequence was also randomized so that each observer was shown the images in a different order. Observers were trained before the study in a pilot study (using 180 image pairs) to become familiar with the software, the different image-processing types, and the task they were being asked to perform. Additional training with a smaller set of 10 image pairs was performed at the beginning of the main study. The study was performed using a web-based interface developed in-house [10]. This interface allowed the study to be performed remotely at two sites simultaneously using the clinical workstation in each site. Each workstation contained a pair of 5-megapixel monitors (site 1: model MDMG-5121, Barco; site 2: model GS521-CL, Eizo) calibrated using the vendors software to the DICOM grayscale standard display function [17]. At both sites, room illuminance was maintained at an ambient level during sessions [18]. Observers were told there could be no lesions, one lesion, or multiple lesions present within each image. They were asked to mark the center of any region that they were suspicious of being TABLE 2: Difference in Reader-Averaged Figure of Merit (FOM) When Readers Are Treated as Fixed and Random Factors Using the ANOVA Technique Image-Processing Pair Being Compared Cancer Type cancerous and were asked not to mark benign features. They were asked to identify if they were marking a calcification cluster, noncalcification cancer, or both. The observer was then asked the following question: What is the likelihood that this lesion is malignant? This question was answered on a 5-point scale, with 1 being the lowest likelihood of malignancy and 5 being the highest likelihood of malignancy. Statistical Analysis For data analysis, cases containing biopsyproven malignant noncalcification cancers and simulated calcification clusters were assumed to be positive for disease. Cases containing biopsyproven benign lesions or normal cases were assumed to be negative for disease. A mark made by an observer had to be within the outline of the cancer made by the expert radiologist to be a correctly localized cancer. All other marks were considered false-positives. The ratings given to each lesion were analyzed using jackknife-alternative free-response receiver operating characteristic (JAFROC) analysis [19]. In this analysis, the performance of an observer inspecting images with a particular imageprocessing algorithm applied is expressed as the figure of merit (FOM) and is calculated as the trapezoidal area underneath the alternative free-response receiver operating characteristic (AFROC) curve when extended to (1,1). This straight line extension credits perfect decisions, such as normal images with no marks, and penalizes unmarked lesions [20]. JAFROC analysis was performed using JAFROC software (JAFROC, version 4.0, D. P. Chakraborty). This analysis performs significance testing using the Dorfman-Berbaum-Metz ANOVA technique [21, 22]; the machine code for the requisite ANOVA module was obtained from the authors of that software package. In the primary analysis, only cases were treated as a random Difference in Reader-Averaged FOM (95% CI) Readers as Fixed Factor Readers as Random Factor Standard vs film-screen Mass ( to 0.029) ( to 0.034) Calcification ( ) a ( ) a Standard vs low contrast Mass ( to 0.031) ( to 0.036) Calcification ( ) a ( to 0.060) Film-screen vs low contrast Mass ( to 0.024) ( to 0.029) Calcification ( to 0.006) ( to 0.020) Note When the 95% CIs do not include zero, there is a significant difference between the image-processing pair. a There is a significant difference between the image-processing pair. factor (so that the results apply to the population of cases but only for the readers used in the study). Secondary analysis was also performed with both cases and readers treated as random factors. For both, a p value < 0.05 was required for significance. An additional analysis compared the number of false-positives marked in images for each type of image processing. The number of false-positives was jack-knifed over cases, and Dorfman-Berbaum-Metz ANOVA was performed. This analysis was first performed using false-positives on all cases. The analysis was then repeated for two subgroups of cases: normal cases and cases with biopsy-proven benign lesions. Finally, the number of times a cancer was correctly localized (with any rating) was determined. Results Noncalcification Cancers The reader-averaged AFROC curves for the noncalcification cancers are shown in Figure 3. The reader-averaged FOMs were equal to (95% CI, ), (95% CI, ), and (95% CI, ) for the standard, low-contrast, and filmscreen image-processing types, respectively. The differences between the reader-averaged FOMs for each pair of image-processing types are given in Table 2; the 95% CIs are shown in parentheses. When the 95% CI of the difference does not include zero, this indicates that there is a significant difference between the two image-processing types. No significant difference was found between any of the three image-processing pairs (p > 0.40) when readers were treated as either a fixed or a random factor in the ANOVA analysis. Calcification Cancers The reader-averaged AFROC curves for calcification clusters are shown in Figure 4. The reader-averaged FOMs were equal to 390 AJR:203, August 2014

5 Image Processing in Digital Mammography TABLE 3: Percentage Difference in Number of False-Positives (FPs) Marked in Different Image-Processing Types Percentage Difference in Number of FPs (95% CI) Image-Processing Pair Being Compared Cancer Type All Cases Normal Cases Cases With Biopsy-Proven Benign Lesions Standard vs film-screen Mass 0.66 ( 8.64 to 7.32) 2.81 ( to 10.39) 7.84 ( 7.24 to 22.92) Calcification ( to 0.20) a ( to 0.88) a ( 6.44 to 27.25) Standard vs low contrast Mass 2.19 ( 9.99 to 5.61) 7.38 ( to 4.85) 2.72 ( to 11.95) Calcification 8.08 ( 8.40 to 24.56) 6.78 ( to 30.43) 1.51 ( to 18.61) Film-screen vs low contrast Mass 2.92 ( 5.11 to 10.95) ( 2.49 to 24.69) 4.67 ( to 9.31) Calcification 9.43 ( 8.61 to 27.56) ( 7.12 to 50.48) ( to 4.48) Note When the 95% CIs do not include zero, there is a significant difference between the image-processing pair. a There is a significant difference between the image-processing pair (95% CI, ), (95% CI, ), and (95% CI, ) for the standard, low-contrast, and film-screen image-processing types, respectively. The differences between the readeraveraged FOMs for each pair of image-processing types are given in Table 2. When readers were treated as fixed factors in the ANOVA analysis, the standard image processing was significantly better than both the film-screen image processing (p = ) and the low-contrast image processing (p = 0.04). There was no significant difference between film-screen image processing and low-contrast image processing (p = 0.15). When readers are treated as random factors in the ANOVA analysis, the difference between standard and low-contrast image processing became nonsignificant; however, the difference between standard and film-screen remained significant (p = 0.03). Additional Analysis The number of false-positive noncalcification marks was not significantly different for any of the image-processing pairs investigated (Table 3). The number of false-positive calcification clusters marks increased by 15% for film-screen image processing compared with standard image processing (p = 0.047), as shown in Table 3. When analyzing the subgroup of cases containing biopsy-proven benign lesions, we found that there was no significant difference in the number of false-positive calcification cluster marks for any image-processing pair (p > 0.16). For the subgroup of images with normal findings, the number of false-positive calcification cluster marks was significantly lower for standard image processing than for filmscreen image processing (p = 0.041). The number of times each cancer was correctly localized (mark made within outline of cancer made by expert radiologist) was calculated. The maximum number of times a cancer can be localized is 21 (seven observers reading all three image-processing types). The proportion of times the cancers were localized by all the observers in all the image-processing types and the number of times the cancer was not localized at all are given in Table 4. Discussion We investigated the effect of image processing on the detection of calcification clusters and noncalcification cancers. We found Lesion Localization Fraction Standard IP 0.2 Low-contrast IP Film-screen IP False-Positive Fraction Fig. 3 Reader-averaged alternative free-response receiver operating characteristic curves for noncalcification cancers for three different imageprocessing (IP) types. Empirical curves are shown. Solid lines = measured data, dashed lines = straight line extension to (1,1). that image processing had a significant effect on the detection of calcification clusters but not noncalcification cancers. Several studies have investigated the effect of image processing on breast cancer detection in general, including both calcifications and noncalcification cancers together [4, 6], or on one cancer type only [5, 9]. We analyzed the noncalcification cancers and calcification clusters separately to determine whether image processing had a different effect on the detection of the different radiologic features. Cole et al. [8] and Kamitani et al. [7] also analyzed TABLE 4: Percentage of Cancers Not Correctly Localized by Any of the Seven Observers in Any of the Three Image-Processing Types Investigated and Percentage of Cancers Seen by All Seven Observers in All Three Image-Processing Types Cancers Noncalcification Cancers (%) Calcification Clusters (%) Cancers not localized at all 1 16 Cancers localized every time 26 6 Lesion Localization Fraction Standard IP Low-contrast IP Film-screen IP False-Positive Fraction Fig. 4 Reader-averaged alternative free-response receiver operating characteristic curves for calcification clusters for three different imageprocessing (IP) types. Empirical curves are shown. Solid lines = measured data, dashed lines = straight line extension to (1,1). AJR:203, August

6 Warren et al. calcification and noncalcification cancers separately. However, Kamitani et al. used a smaller number of cancer cases than this study, 11 calcification cases and 34 noncalcification cases. Cole et al. also used a much smaller number of cancer cases acquired on a system with an amorphous selenium detector, eight calcification cases and 13 mass cases, compared with the current study and different observers read images with different image-processing algorithms, whereas in this work the same readers interpreted the same cases with different image-processing algorithms to allow a paired comparison. Our work found that the standard imageprocessing algorithm provided significantly higher detection of calcification clusters than the low-contrast and film-screen image-processing algorithms. The results of this study are in agreement with the work by Zanca et al. [5]; they performed a study with a design similar to the present work and found significant differences in calcification detection between several pairs of image-processing algorithms. However, Warren et al. [9], who also performed a study of similar design, found no significant differences in detection between two different image-processing algorithms. Given that the designs of our study and these two studies were similar, the differences in results may be because of differences in the image-processing algorithms investigated. Our work also found that there were no significant differences in detection of noncalcification cancers between the different types of image processing. This result is in agreement with a study by Cole et al. [8], which found no significant difference in the area under the receiver operating curve for masses. However, that study was limited to small numbers of cases, as described earlier. In this study, we considered marks made by observers on biopsy-proven benign lesions as false-positives. This approach has been used elsewhere [23] and assumes the ground truth is based on biopsy-proven cancers. If, instead, we consider that biopsyproven benign lesions are not false-positives because a radiologist must have recalled this patient originally for a biopsy to be performed (correct recall) and the results are reanalyzed, the same conclusions are reached. Visser et al. [6] found that image processing affected the suspiciousness of normal and malignant features in the breast. In this work we found that there was no significant difference in the number of false-positives with changes in the image processing for noncalcification cancers. However, for calcification clusters there was a significant decrease in the number of false-positives reported on normal images for standard image processing compared with film-screen image processing. Some of these images showed benign features that were not recalled for biopsy in the clinic. The change in the number of false-positives in these images indicates that benign features that did not appear suspicious enough to be recalled in the clinic can appear suspicious enough to be recalled when a different type of image processing has been applied. The biggest limitation to this work is the number of cases and readers. There was large variability among readers, as shown by the wide 95% CI for the reader-averaged FOM for each image-processing type. In addition, a proportion of the noncalcification cancers were seen by all observers in all image-processing algorithms (Table 4). These lesions may have been too obvious to detect possible differences among image-processing algorithms. A proportion of the calcifications were not seen at all (Table 4). These lesions were too subtle to be useful in the study. Therefore, in retrospect this study would have had more power if some more subtle noncalcification cancers and if some more obvious calcifications had been included in the image set and if more observers had been enrolled in the study. There is a limit to the number of readers it is practical to enroll in such studies; therefore, the power limitations of this study emphasize the need for careful case selection when preparing observer studies. In summary, for the particular image-processing algorithms and systems investigated, we found that the standard image processing was superior for the detection of calcifications and that there was no significant difference among image-processing algorithms for noncalcification lesions. The differences measured in the JAFROC FOM for calcification detection were between 4% and 6%. This difference is smaller than the change in calcification detection for other factors such as detector type or dose [9]. Finally, the results in this study are applicable for the particular image-processing algorithms and systems investigated. We recommend that objective measurements, such as the method used in this study, be applied when selecting the optimal image processing to be used by radiologists with other imageprocessing algorithms and systems. Acknowledgments We thank Carole Kliger, Caroline Taylor, Victoria Cooke, Louise Wilkinson, Wilmi Pienaar, Rupika Mehta, and Christine Flis for taking part in the study. We also thank Lindsay Mungutroy and Charul Patel for their assistance in running the observer study. Thanks are also due to Claire Borrelli, Sharon Montaque, and Sue Storr for organizing time on the workstations to complete the study. We thank David Duncan for writing the software used to insert the calcification clusters into breast images. We are grateful to the staff at King s College Hospital in particular, Asif Iqbal and Michael Michell for their assistance collecting breast images. We are also grateful for the help and cooperation of staff from the Jarvis Breast Screening Centre while collecting breast images for the study. Finally, we are grateful to Hologic staff for advice and help in processing the images. References 1. Sivaramakrishna R, Obuchowski NA, Chilcote WA, Cardenosa G, Powell KA. Comparing the performance of mammographic enhancement algorithms: a preference study. AJR 2000; 175: Pisano ED, Cole EB, Major S, et al. Radiologists preferences for digital mammographic display. Radiology 2000; 216: Goldstraw EJ, Castellano I, Ashley S, Allen S. The effect of Premium View post-processing software on digital mammographic reporting. Br J Radiol 2010; 83: Chen B, Wang W, Huang J, et al. Comparison of tissue equalization, and premium view post-processing method in full field digital mammography. Eur J Radiol 2010; 76: Zanca F, Jacobs J, Van Ongeval C, et al. Evaluation of clinical image processing algorithms used in digital mammography. Med Phys 2009; 36: Visser R, Veldkamp WJH, Beijerinck D, et al. Increase in perceived case suspiciousness due to local contrast optimisation in digital screening mammography. Eur Radiol 2012; 22: Kamitani T, Yabuuchi H, Soeda H, et al. Detection of breast cancer by soft-copy reading of digital mammograms: comparison between a routine image-processing parameter and high-contrast parameters. Acta Radiol 2010; 51: Cole EB, Pisano ED, Kistner EO, et al. Diagnostic accuracy of digital mammography in patients with dense breasts who underwent problem-solving mammography: effects of image processing and lesion type. Radiology 2003; 226: Warren LM, Mackenzie A, Cooke J, et al. Effect 392 AJR:203, August 2014

7 Image Processing in Digital Mammography of image quality on calcification detection in digi- calcifications for observer studies in digital mam- cal imaging systems: executive summary of AAPM tal mammography. Med Phys 2012; 39: mography. Phys Med Biol 2013; 58:N217 N228 TG18 report. Med Phys 2005; 32: Looney PT, Halling-Brown M, Given-Wilson R, et 15. Zanca F, Zhang G, Marshall N, et al. Software 19. Chakraborty DP, Berbaum KS. Observer studies in- al. A national digital mammography image re- framework for simulating clusters of microcalcifi- volving detection and localization: modelling, analy- source. (abstract) In: British Institute of Radiology. cations in digital mammography. In: Martí J, Oli- sis and validation. Med Phys 2004; 31: Symposium Mammographicum Conference ver A, Freixenet J, Martí R, eds. Digital mammog- 20. Chakraborty DP. A brief history of FROC para- London, UK: British Institute of Radiology, 2012: Bennett RL, Sellars SJ, Moss SM. Interval cancers in the NHS Breast Cancer Screening Programme in England, Wales and Northern Ireland. Br J Cancer 2011; 104: Ruschin M, Timberg P, Båth M, et al. Dose dependence of mass and microcalcification detection in digital mammography: free response human observer studies. Med Phys 2007; 34: Warren LM, Mackenzie A, Dance DR, Young KC. Comparison of the x-ray attenuation properties of breast calcifications, aluminium, hydroxyapatite and calcium oxalate. Phys Med Biol 2013; 58:N103 N Warren LM, Green FH, Shrestha L, Mackenzie A, Dance DR, Young KC. Validation of simulation of raphy: 10th International Workshop, IWDM 2010, Girona, Catalonia, Spain, June 16 18, 2010 proceedings. Berlin, Germany: Springer- Verlag, 2010; 6136: Liston J, Wilson R, eds. Quality assurance guidelines for breast cancer screening radiology, 2nd ed. Sheffield, England: NHS Breast Screening Programmes, 2011: publication no National Electrical Manufacturers Association. Digital imaging and communications in medicine (DICOM). Part 14. Grayscale standard display function. Rosslyn, VA: National Electrical Manufacturers Association, 2000: publication no. PS Samei E, Banano A, Chakraborty D, et al.; AAPM TG18. Assessment of display performance for medi- digm data analysis. Acad Radiol 2013; 20: Dorfman DD, Berbaum KS, Metz CE. Receiver operating characteristic rating analysis: generalization to the population of readers and patients with jackknife method. Invest Radiol 1992; 27: Hillis SL, Berbaum KS, Metz CE. Recent developments in the Dorfman-Berbaum-Metz procedure for multireader ROC study analysis. Acad Radiol 2008; 15: Rafferty EA, Park JM, Philpotts LE, et al. Assessing radiologist performance using combined digital mammography and breast tomosynthesis compared with digital mammography alone: results of a multicenter, multireader trial. Radiology 2013; 266: AJR:203, August