Automated Ultrasound of the Breast for Diagnosis: Interobserver Agreement on Lesion Detection and Characterization

Transcription

1 Women s Imaging Original Research Shin et al. Automated Breast Ultrasound Women s Imaging Original Research Hee Jung Shin 1 Hak Hee Kim Joo Hee Cha Ju Hyun Park Kyoung Eun Lee Jeoung Hyun Kim Shin HJ, Kim HH, Cha JH, Park JH, Lee KE, Kim JH 1 All authors: Department of Radiology and Research Institute of Radiology, Asan Medical Center, University of Ulsan, College of Medicine, 86 Asanbyeongwon-gil, Songpa-gu, Seoul, , South Korea. Address correspondence to H. H. Kim (hhkim@amc.seoul.kr). AJR 2011; 197: WOMEN S IMAGING Keywords: automated ultrasound, breast, neoplasm DOI: /AJR Received September 24, 2010; accepted after revision March 7, X/11/ American Roentgen Ray Society Automated Ultrasound of the Breast for Diagnosis: Interobserver Agreement on Lesion Detection and Characterization OBJECTIVE. The purpose of this study was to prospectively evaluate interobserver agreement on lesion detection and characterization in the review of automated ultrasound images of the breast by five radiologists. SUBJECTS AND METHODS. From August to October 2009, bilateral whole-breast ultrasound examinations were performed with an automated technique and with a handheld device for 55 women consecutively scheduled to undergo diagnostic ultrasound. Three-dimensional volume data from automated ultrasound were reviewed by five radiologists, who were unaware of the results of ultrasound with a handheld device and mammography and of the clinical information. If a lesion was detected with automated ultrasound, clock-face position, distance from the nipple, largest diameter, and BI-RADS final assessment category were evaluated. If the lesion was a mass, shape, orientation, margin, echogenicity, and posterior feature were analyzed. Intraclass correlation coefficients and kappa statistics were used for statistical analysis. RESULTS. At least two observers identified 145 lesions with automated ultrasound. Among 725 possible detections, 587 (81%) detections were made. Individual investigators detected between 74% (107/145) and 88% (127/145) of the lesions. The rate of detection of lesions larger than 1.2 cm was 92%. Most lesions detected only with handheld ultrasound (11/12, 92%) or automated ultrasound (34/36, 94%) were cysts or probably benign masses. All intraclass correlation coefficients for lesion location and size exceeded 0.75, indicating high reliability. Substantial agreement was found for mass shape (κ = 0.71), orientation (κ = 0.72), margin (κ = 0.61), and BI-RADS final assessment category (κ = 0.63). CONCLUSION. Detection of lesions larger than 1.2 cm in greatest diameter was reliable. High reliability was obtained for reporting lesion size and location. Substantial agreement was obtained for description of key feature and final assessment category. B reast ultrasound has become accepted as a diagnostic tool to be used in conjunction with mammography for evaluating breast abnormalities [1]. The use of breast ultrasound preoperatively and in interventional procedures is well established [2, 3], and the use of mammography as the standard method for breast cancer screening has reduced breast cancer mortality [4]. Because mammograms are summation images, all breast tissue overlapping in each view, the value of mammography for cancer detection is reduced in the care of women with dense breast tissue, and mammographically missed cancer is a particular problem among these women [5, 6]. Boyd et al. [6] found that the odds of interval cancer among women with extremely dense breasts was 17.8 times that among women with fatty breasts. Ultrasound is an attractive supplement to mammography because it is widely available, relatively inexpensive, and well-tolerated by patients [7]. The combined diagnostic yield of mammography and breast ultrasound has been found to be greater than that of mammography alone [8] and is reported to be % among women with palpable masses [9]. Bilateral whole-breast ultrasound may also have value. Ultrasound can be used to identify additional invasive cancer and help assess the extent of disease in women with newly diagnosed cancer [2, 10 12]. Early results of single-center studies have been promising for cancer detection [13 17], but the time and skill necessary to detect small nonpalpable tumors with handheld ultrasound devices has discouraged widespread use [18, 19]. The results of a multicenter trial [20, 21] AJR:197, September

2 Shin et al. of supplemental screening breast ultrasound for women at high risk with dense breast tissue have been promising, but lack of uniformity and shortage of qualified personnel limit wide implementation of ultrasound with a handheld device. Using automated whole-breast ultrasound, personnel with lesser training can gather standardized uniform images, and physicians can interpret the image data sets efficiently in a short time. Several previous studies have shown that 3D automated breast ultrasound is feasible [22, 23] and that it facilitates detection and classification of solid and cystic lesions with high sensitivity in a selected patient group [24]. Kelly et al. [25, 26] found that using automated breast ultrasound in addition to mammography improved the accuracy of breast cancer detection, callback rates, and confidence in callbacks of women with dense breast tissue. Consistent recognition and characterization of lesions is critical in whole-breast ultrasound applications. They require consistent reporting of lesion location, size, and description and depiction of features. The purpose of our study was to prospectively evaluate interobserver agreement on lesion detection and characterization among five breast radiologists reviewing automated breast ultrasound images. Subjects and Methods Participants Our institutional review board approved this study. From August to October 2009, study participation was offered to patients who underwent consecutive diagnostic ultrasound examinations with handheld devices at our institution. Patients who agreed to participate in the study and signed an informed consent form were recruited for this study. Women with greater than 7-cm compressed breast thickness at mammography were specifically not recruited because of the limited effectiveness of ultrasound in identifying small masses at this depth and the resultant decreased reliability of automated ultrasound of the breast in examinations of these women [25]. A final sample of 55 patients (age range, years; mean, 48 years) underwent automated and handheld ultrasound in the supine position. The indications for ultrasound were preoperative staging of breast cancer for 20 patients, inconclusive mammographic findings for 10 patients, workup for abnormalities found at breast ultrasound at other hospitals for 10 patients, evaluation of palpable lump for six patients, follow-up of probably benign findings for six patients, nipple discharge for two patients, and follow-up after breast cancer surgery for one patient. Breast Ultrasound Handheld device Breast ultrasound with the handheld device (IU22, Philips Healthcare) equipped with a 50-mm linear-array transducer with a bandwidth of 5 12 MHz was performed by one of five radiologists with 1, 3, 6, and 9 years of experience according to a standardized scanning protocol. At our institution, the scanning technique for bilateral whole-breast ultrasound is standardized as follows: scanning of the right breast in the transverse and sagittal orientations, scanning of the inner aspect of the breast in a supine position, and scanning of the outer aspect of the breast in a supine oblique position with the woman s arm raised above her head. We documented each cystic and solid lesion with an image of its largest horizontal diameter (recording both horizontal and vertical diameters) and an image perpendicular to that with its respective diameter. For each lesion, we recorded the location according to breast, clock position to the nearest half hour, and estimated distance from the nipple in centimeters. The average time to perform a ultrasound examination with a handheld device was approximately 20 minutes. Automated system Automated ultrasound of the breast (Acuson S2000 automated breast volume scanner, Siemens Healthcare) was performed by two technologists, who had participated in 2 weeks of training in the technique. The device consisted of the automated breast volume scanner module with the core components of flexible arms, touchscreen monitor, and scanner (transducer, scan box, and screen membrane for contact). The large-footprint wide-frequency-bandwidth transducer (5 14 MHz with a 9-MHz center frequency) captures a volume of up to 15.4 cm 16.8 cm 6 cm maximum by acquisition of a series of 320 high-resolution axial 2D images at slice intervals of 0.5 mm. For scanning with the automated system, customized presets were used to optimize depth, gain, frequency, and view. A typical examination comprised three automated scans of each breast in the anteroposterior and both oblique positions. Occasional additional views were required for larger breasts, the scans being centered on a palpable abnormality or axillary lymph nodes. After acquisition, proprietary postprocessing algorithms were applied according to nipple location to maximize the quality of the diagnostic information. The proprietary postprocessing algorithms included a reverberation removal algorithm, an adaptive nipple shadow reduction tool, and a gain-correction algorithm provided and developed by the manufacturer. After acquisition, the series of axial images was automatically sent from the automated breast volume scanner to a dedicated breast ultrasound review workstation. The usual acquisition time for automated ultrasound of the breast was 60 seconds per scan. The total acquisition time per patient, including setup time, was 15 minutes. The system captured the volume data at slice intervals of 0.5 mm. After volume data acquisition, the data were automatically sent from the automated breast volume scanner to the workstation and reviewed in multiple orientations in a multiplanar reconstruction display. A total of 320 images were generated at 0.5-mm slice thickness and 160 images at 1-mm slice thickness. Therefore, at three scans per breast, 960 images were obtained for each breast at 0.5-mm intervals and 480 images at 1-mm intervals. Spatial compounding was applied during the scanning process, and harmonic imaging was feasible but not used for automated ultrasound. Features Recorded Five radiologists with 1, 3, 6, and 9 years of experience evaluated the 3D volume data at the automated breast ultrasound workstation. They had performed and interpreted images from at least 1500 breast ultrasound examinations in the previous year. They participated in a 3-hour tutorial explaining operation of the automated breast ultrasound review workstation, and all readers reviewed and discussed approximately 10 automated breast ultrasound examinations. The radiologists were blinded to the findings on the corresponding mammograms and handheld ultrasound images and to clinical information. Each reader evaluated automated ultrasound data according to the BI-RADS lexicon [27]. The BI-RADS ultrasound [27] features were recorded, beginning with whether the lesion was a special case. Special cases were defined as any of the following; cyst, complicated cyst, clustered microcysts, intraductal mass, lymph node, postsurgical scar, or calcifications. For each lesion, each reader was required to make a BI-RADS final assessment as one of the following: 1, negative; 2, benign; 3, probably benign (2% or lower probability of malignancy); 4A, low suspicion (risk of malignancy, 3 10%); 4B, intermediate suspicion (risk of malignancy, 11 49%); 4C, moderate suspicion (risk of malignancy, 50 94%); or 5, highly suggestive of malignancy (95% or greater likelihood of malignancy). For cases that were not deemed special, the following additional features were recorded: shape (oval, round, or irregular), orientation (parallel to the skin surface or not), margin (circumscribed, microlobulated, indistinct, spiculated, angular), boundary (abrupt or echogenic halo), echo pattern (anechoic, isoechoic, hyperechoic, hypoechoic, mixed hyperechoic-hypoechoic, complex cystic), and posterior feature (none, enhancement, shadowing, or combined enhancement and shadowing). Finally, each reader recorded lesion location (clock-face position, distance from the nipple in 748 AJR:197, September 2011

3 Automated Breast Ultrasound centimeters), largest diameter in centimeters, BI- RADS features [27], and BI-RADS final assessment category. Ultrasound findings of benign lesions included cyst, fat lobule, fibrous ridge, ductal ectasia, and hyperechoic lesion. Probably benign lesions were defined as circumscribed, oval, or gently lobulated nonpalpable hypoechoic masses and had no suspicious findings. Suspicious lesions were defined as noncircumscribed hypoechoic masses with a nonparallel orientation, microcalcifications, posterior shadowing, echogenic halo, or ductal extension. HHUS: 121 lesions ABUS only: 36 lesions Exclusion: 12 lesions detected by only one reader on ABUS 145 lesions in 55 patients detected by at least two readers Malignant: 28 lesions High-risk: 3 lesions Benign or probably benign: 114 lesions Data Review and Statistical Analysis The primary aim of this study was to estimate the reproducibility of assessment of various lesion characteristics across multiple readers using automated breast ultrasound for diagnostic purposes. The assessments included measurement of size, identification of lesions, and recording of lesion location. Initially, summary tables and simple frequencies were used to explore the data and check for outliers. Intraclass correlation coefficient was calculated for continuous variables, such as number of lesions detected, lesion size, clock position, and distance from the nipple, compared with the findings with handheld ultrasound. Kappa statistics were used to measure agreement on lesion features and final assessments compared with the consensus finding. Missing values were excluded. A kappa value of 1.0 denoted perfect agreement; , almost perfect agreement; , substantial agreement; , moderate agreement; , fair agreement; and 0.20 or less, slight agreement [28]. Two-sided 95% CI was used to estimate the intraclass correlation coefficient. Agreement values were estimated for different subgroups according to various parameters (e.g., size, benign vs malignant histopathologic outcome). For size, we arbitrarily determined a cutoff point of 0.7 cm, which was the median value of the longest diameter. For categoric variables, we pooled data from all five readers to obtain overall percentages. For data analysis, the findings at automated ultrasound and handheld ultrasound were compared by one radiologist (6 years of experience in breast ultrasound), and lesions were matched and assigned lesion numbers across five readers. To reduce the effect of false-positive detection, for reproducibility and kappa statistics, we considered only lesions seen by at least two readers. The consensus description across all readers identifying any given lesion was considered the reference standard for that lesion. Consensus was defined as detection of a lesion by at least two readers. Fig. 1 Flow chart summarizes study sample in terms of numbers of lesions. ABUS = automated breast ultrasound, HHUS = handheld ultrasound. Results Participants A total of 121 lesions in 55 patients (age range, years; mean, 48 years) were identified with conventional handheld ultrasound. Thirty-six lesions were additionally identified with automated breast ultrasound. Four of the 157 lesions were detected by no reader, and eight lesions were detected by only one reader using automated ultrasound. Therefore, 145 lesions were identified by at least two observers using automated ultrasound, and these 145 lesions comprised the study sample (Fig. 1). The largest diameter of the 145 lesions ranged from 0.2 to 11.0 cm (mean, 1.1 ± 1.4 [SD] cm; median, 0.7 cm). The largest diameter of the 28 malignant lesions ranged from 0.6 to 9.3 cm (mean, 2.4 ± 1.7 cm; median, 2.1 cm). Twenty patients had 28 malignant lesions and underwent 22 ultrasound-guided biopsies. Seventeen patients had invasive ductal carcinoma, one patient had ductal carcinoma in situ, one patient had invasive lobular carcinoma, and one patient had metaplastic carcinoma. Three patients had high-risk lesions diagnosed at subsequent percutaneous biopsy, all of which were papilloma. All papillomas were excised. The other 32 patients had benign or probably benign lesions according to the review of mammograms and handheld ultrasound images. Lesion Detection With Automated Whole-Breast Ultrasound Of 725 potential detections, 587 (81%) detections were made. Individual investigators detected between 107 (74%) and 127 (88%) of the 145 lesions in the sample. Twentythree (16%) were detected by only two investigators; 22 (15%) were detected by three investigators; 25 (17%) were detected by four investigators; and 75 (52%) were detected by five investigators. Among a total of 138 detections of 70 lesions missed by fewer than three readers, 70 (51%) were simple (n = 45) or complicated cysts (n = 25) with an average size of 0.5 cm; seven (5%) were ductal ectasia; 50 (36%) were probably benign masses with an average size of 0.7 cm; six (4%) were indeterminate masses with no proven malignancy; and five (4%) were malignant lesions proved pathologically that were missed by one reader (one lesion) or two readers (two lesions) at automated ultrasound. With respect to the 12 lesions identified with handheld ultrasound and not detected with automated ultrasound (four by none of the five radiologists, eight by only one radiologist), five corresponded to cysts, four to probably benign masses (diameter, 0.7 cm or less), two to ductal ectasia, and one to biopsyproven ductal carcinoma in situ (Fig. 2). This lesion was very subtle even on ultrasound images obtained with the handheld device. The radiologist who performed handheld ultrasound reviewed mammograms at the time and detected this subtle lesion, which corresponded to pleomorphic calcifications on mammograms. However, none of the five readers detected this subtle lesion with automated ultrasound (Fig. 2). The other three lesions not detected by any of the five readers were one cyst and two probably benign masses. Pathologic examination showed 20 patients had 28 biopsy-proven malignant lesions. Twenty-four of the 28 malignant lesions were detected by all five readers, and three lesions were detected by three or four readers (Fig. 3). The single remaining lesion was not detected by any of the five readers, as mentioned earlier (Fig. 2). Among 36 lesions detected only with automated ultrasound, 20 lesions were considered consensus masses and 16 lesions were AJR:197, September

4 Shin et al. considered consensus cysts. Eighteen consensus masses were interpreted as probably benign, and the other two were interpreted as indeterminate. After automated ultrasound, these two patients underwent second-look ultrasound. One lesion was not reproducible at handheld ultrasound and was misinterpreted by two readers at automated ultrasound owing to the presence of Cooper ligament shadowing (Fig. 4). The other lesion was considered probably benign at handheld ultrasound. The lesion detection rate increased as lesion size increased, as follows: 107 of 155 (69%) potential detections were made at a lesion size of 0.4 cm or smaller, 243 of 310 (78%) at cm, 66 of 75 (88%) at 0.9 A 1.2 cm, and 171 of 185 (92%) at more than 1.2 cm in diameter. The sensitivity and specificity of handheld ultrasound were 100% and 93%. The sensitivity of automated ultrasound for the five readers was 100%, 90%, 100%, 93%, and 97%, and the specificity was 92%, 91%, 86%, 96%, and 94%. The area under the curve was for handheld ultrasound: for reader 1, for reader 2, for reader 3, for reader 4, and for reader 5. Agreement The intraclass correlation coefficients for number of detected lesions, clock position, distance from the nipple, and largest diameter were 0.83 (95% CI, ), 0.77 (95% CI, ), 0.89 (95% CI, ), and 0.92 (95% CI, ). All exceeded 0.75, which indicated very high reliability. In terms of comparison of the findings with handheld ultrasound and automated ultrasound, intraclass correlation coefficients for clock position, distance from the nipple, and largest diameter were 0.75, 0.89, and Sixty-five of the 145 lesions were considered special cases. For special cases (cyst, complicated cyst, ductal ectasia, calcifications, lymph node, or not a special case), the overall kappa value was 0.63 (Table 1). Specific designation of a lesion as a cyst or complicated cyst had kappa values of 0.65 and Special cases were excluded from subsequent feature analysis, which left 80 solid masses to be described by consensus. For mass, the overall kappa value was 0.75 (Table 1). Substantial agreement was found for assessment of lesion shape (κ = 0.71) and mass orientation (κ = 0.72). Reader agreement was greatest for irregular shape (κ = 0.76) and lowest for round shape (κ = 0.25) (Table 2). Substantial agreement was found for margin assessment (κ = 0.61). Almost perfect agreement was found when margins were grouped into circumscribed or not (κ = 0.808). Moderate agreement was found between readers in echogenicity assessment (κ = 0.45). The lowest agreement was noted for complex cystic masses (κ = 0.33). Fig year-old woman with biopsy-proven cancer of right breast. A, Mammogram of left breast shows pleomorphic calcifications with segmental distribution in upper outer quadrant. B, Handheld ultrasound image of left breast shows subtle mixed hyperechoic-hypoechoic lesions with suspicious echogenic foci (arrows). C, Automated ultrasound image of left breast shows subtle localized areas of heterogeneous echogenicity (arrows) without definite associated mass in upper outer quadrant that were not detected by any of five readers. Surgery confirmed presence of 3.0-cm ductal carcinoma in situ, micropapillary pattern. B A C B Fig year-old woman with breast cancer. A, Handheld ultrasound image shows irregularly shaped hypoechoic mass with microlobulated margin (arrows) adjacent to biopsy-proven malignant mass (circle) in upper outer quadrant of left breast. B, Automated ultrasound image shows malignant-looking mass (arrows) adjacent to biopsy-proven malignant mass (circle), which was not detected by one reader. Surgery confirmed two foci of invasive ductal carcinoma. 750 AJR:197, September 2011

5 Automated Breast Ultrasound TABLE 1: Interobserver Agreement on BI-RADS Descriptors BI-RADS Descriptor No. of Lesions k Subgroup k Special case (0.040) Cyst (0.05) Complicated cyst (0.07) Duct ectasia 3 NA Calcifications 1 NA Lymph node 1 NA Mass (0.04) Note Data in parentheses are standard error. NA = not applicable (no entries). TABLE 2: Interobserver Agreement on BI-RADS Features of 80 Masses A B Fig year-old woman with indeterminate breast mass. A, Automated ultrasound image of left breast shows irregularly shaped hypoechoic mass (arrow) with posterior shadowing that was interpreted as indeterminate by two readers. B, Second-look handheld ultrasound image shows lesion (arrow) was not reproducible. Finding was considered shadowing due to Cooper ligament. Moderate agreement was achieved in description of posterior acoustic features (κ = 0.42). BI-RADS final assessment categories of the 145 lesions were reported as 1 or 2 (40%, 240 of 603), 3 (31%, 184 of 603), 4A (11%, 68 of 603), 4B (3%, 19 of 603), 4C (2%, 12 of 603), and 5 (13%, 80 of 603) by five readers (Table 3). Examination of all categories together showed substantial agreement (κ = 0.63). Substantial agreement was found when final assessment was grouped into three categories: 1 and 2, 3, and 4A 4C and 5 (κ = 0.712). In terms BI-RADS Descriptor No. of Interpretations (n = 358) a k b Subgroup k b Mass shape 0.71 (0.06) Round 12 (3) 0.25 (0.11) Oval 211 (59) 0.74 (0.05) Irregular 135 (38) 0.76 (0.06) Mass orientation 0.72 (0.08) Mass margins 0.61 (0.04) Circumscribed 195 (54) 0.81 (0.05) Microlobulated 38 (11) 0.20 (0.06) Indistinct 75 (21) 0.68 (0.08) Spiculated 24 (7) 0.52 (0.15) Angular 26 (7) 0.11 (0.06) Lesion boundary 0.43 (0.07) Echogenicity relative to fat 0.45 (0.05) Isoechoic 107 (30) 0.42 (0.05) Hypoechoic 227 (63) 0.48 (0.06) Mixed hyperechoic-hypoechoic 10 (3) 0.49 (0.20) Complex cystic 14 (4) 0.33 (0.05) Posterior features 0.42 (0.07) None 276 (77) 0.59 (0.07) Shadowing 52 (15) 0.47 (0.09) Enhancement 30 (8) 0.28 (0.06) Note 358 interpretations of 80 masses. Data are from automated ultrasound across five observers. NA = not applicable (no entries). a Data in parentheses are percentages. b Data in parentheses are standard error. AJR:197, September

6 Shin et al. TABLE 3: Percentage of Lesions in Each BI-RADS Category and Interobserver Agreement BI-RADS Final Assessment Category or Subdivision Percentage of Interpretations (n = 603) k a Subgroup k 0.63 (0.04) 1 or (0.07) (0.05) 4A (0.07) 4B (0.03) 4C (0.05) (0.07) Note 603 interpretations of 145 lesions based on automated ultrasound findings. Data are for five observers. Data in parentheses are standard error. a For grouped final assessments (1 or 2, 3, and 4A C or 5), k = of comparison of the findings at handheld and automated ultrasound, substantial agreement also was found for final assessment (κ = 0.64). Subgroup Findings With respect to malignant versus benign masses, reader agreement was lower regarding malignant lesions in the assessment of mass orientation and posterior acoustic feature (Table 4). Agreement on other features, such as mass shape, margin, lesion boundary, and echogenicity, was greater for malignant lesions than for benign lesions. With respect to grouping of the lesions on the basis of size, reader agreement was generally less for lesions 0.7 cm and smaller. This difference between size groups was more pronounced for mass shape, margin, echogenicity, and posterior acoustic feature. Similarly, worse concordance on assessment of BI-RADS category was noted for masses measuring 0.7 cm and smaller (κ = 0.37) than for masses larger than 0.7 cm (κ = 0.67). Discussion Breast ultrasound is considered an invaluable tool in breast imaging and a first-line examination with a role in both detection and characterization of breast lesions [14]. As TABLE 4: Interobserver Agreement on BI-RADS Descriptors According to Size and Nature of Mass are other breast imaging techniques, breast ultrasound is affected by a lack of reproducibility in lesion characterization, particularly of small lesions [28 34]. Unlike lesions detected at most other radiologic examinations, a lesion not detected during breast ultrasound generally is not documented. Automated whole-breast ultrasound has potential for complete documentation. It is well known that compared with use of mammography alone, the use of breast ultrasound improves breast cancer detection [19, 25, 35]. However, the operator dependence of handheld ultrasound is a major concern with respect to the widespread use of whole-breast ultrasound. Automated ultrasound has several advantages over handheld ultrasound: The technique is more readily reproducible, has 3D capability through multiplanar reconstruction, and allows delayed interpretation outside of real time, optimizing the radiologist s reading environment. When we interpret automated ultrasound images in routine clinical practice, consistent recognition of lesions, especially breast cancer, across multiple readers is an important issue. In our study, we found high reliability in the recording of lesion location, which was described as lesion size, clock position, and distance from the nipple in centimeters. Previous studies showed that 6.6% of women needed short-interval follow-up of probably benign findings seen only at ultrasound [35]. Our results suggest that such follow-up with automated ultrasound is feasible because lesion size and location are consistently recorded. The primary goal of our study was assessment of the reliability of automated ultrasound for lesion detection, description, and interpretation in the diagnostic setting. We found the rate of lesion detection increased as lesion size increased; detection was reliable (92%) only when mean lesion diameter was greater than 1.2 cm. In terms of lesion detection, 12 of 121 lesions detected with handheld ultrasound were not detected by all five readers (four lesions) or were detected by only one reader (eight lesions). Only one of these 12 cases was biopsy-proven ductal carcinoma in situ (Fig. 2), which was a very subtle lesion even at handheld ultrasound. In routine practice, radiologists review mammographic and clinical findings before interpreting automated ultrasound images. The problem of missing small subtle lesions in routine practice can be avoided when automated ultrasound findings are interpreted along with the clinical and mammographic findings. In addition, all but one of the Size Nature Descriptor Overall (n = 80) 0.7 cm (n = 18) > 0.7 cm (n = 62) Benign (n = 50) Malignant (n = 30) Mass shape 0.71 (0.06) 0.35 (0.16) 0.75 (0.06) 0.40 (0.12) 0.64 (0.12) Mass orientation 0.72 (0.08) 0.68 (0.10) 0.72 (0.09) 0.71 (0.09) 0.69 (0.19) Mass margin 0.61 (0.04) 0.40 (0.14) 0.67 (0.05) 0.40 (0.06) 0.65 (0.08) Lesion boundary 0.43 (0.07) 0.32 (0.07) 0.54 (0.07) 0.45 (0.07) 0.52 (0.13) Echogenicity 0.45 (0.05) 0.31 (0.11) 0.55 (0.05) 0.42 (0.09) 0.47 (0.07) Posterior acoustic feature 0.42 (0.07) 0.30 (0.12) 0.52 (0.07) 0.45 (0.09) 0.41 (0.08) BI-RADS final assessment category 0.63 (0.04) 0.37 (0.16) 0.67 (0.05) 0.55 (0.07) 0.65 (0.09) Note Data in parentheses are standard error. 752 AJR:197, September 2011

7 Automated Breast Ultrasound 28 biopsy-proven malignant lesions (the exception was the aforementioned carcinoma in situ) were detected by more than three readers, which showed high reliability of cancer detection with automated ultrasound. Falsepositive detection is also an issue with respect to incorporating automated ultrasound in routine practice. In this study, 34 of 36 (94%) lesions detected only with automated ultrasound were small probably benign masses or cysts, and only two (6%) lesions were interpreted as consensus indeterminate masses. These findings were not reproducible at handheld ultrasound. The false-positive detection rate (6%) leading to possible recommendation of biopsy based on automated ultrasound findings was sufficiently low. In diagnostic applications, it is critical to minimize the rate of performance of unnecessary biopsy. Accurate lesion characterization is critical to successful use of automated ultrasound. We found substantial agreement on description of special cases (κ = 0.63) and masses (κ = 0.75) and relatively inconsistent characterization of complicated cysts. Berg et al. [33] reported that detection and characterization of cysts varied with lesion size and that characterization of lesions as simple cyst equivalents was unreliable for lesions 0.3 cm in diameter or smaller. At least some of the variability in differentiating simple cysts from complicated cysts and solid lesions likely relates to variable user-defined gain and possibly difference in scanning pressure. However, because we are familiar with automated ultrasound images and can change the contrast, this problem can be ameliorated. In terms of mass shape and margin, we found a higher rate of agreement on circumscribed margin but a lower rate on angular margin and round shape compared with results of a previous study [34]. This difference can be explained by the small number of lesions with angular margins (seven lesions) and a round shape (three lesions). On the other hand, the proportion of probably benign masses in our study was larger than that in the previous study, which might have contributed to the higher rate of agreement on circumscribed margins. The differences from findings in other studies also might have been related to the small sample size and the differences between the 3D volume data acquired with automated ultrasound and the 2D data acquired with handheld ultrasound. In other words, we reviewed 3D volume data using multiplanar images, whereas in previous studies of handheld ultrasound, the investigators reviewed representative 2D captured images. Several previous studies [29, 31, 32, 34] have shown substantial agreement on key features of lesion description at handheld ultrasound. We found moderate agreement on lesion boundary, echogenicity, and posterior acoustic features. In subgroup analysis, a higher rate of agreement was found for lesions with a diameter greater than 0.7 cm than for the smaller lesions. A higher rate of agreement also was found for malignant than for benign lesions with respect to key features such as mass shape, margin, and echogenicity. Management according to BI-RADS final assessment is important and should be reproducible across readers. We found substantial agreement with a kappa value of 0.63 for BI-RADS final assessment, which is better than the results of previous studies [30 32, 34]. Skaane et al. [29] reported slightly lower interobserver agreement on management based on ultrasound than mammographic or combined readings with a mean kappa value of 0.48 for hard-copy ultrasound images in comparison with kappa values of 0.58 for mammography and 0.71 for combined readings. Baker et al. [31] reported a kappa value of 0.51 for management based on ultrasound findings. Berg et al. [32] also reported a kappa value of 0.52 for BI-RADS final assessment. In our study, categories 4B and 4C had the lowest kappa values, 0.32 and This finding can be explained by the small number of lesions in these categories and the lack of known factors clearly and objectively defining each subdivision, leading to variable and subjective conceptions of categories 4A, 4B, and 4C among radiologists [34]. In daily practice, automated ultrasound of the breast may have benefit over handheld ultrasound in several respects. First, patients with multiple masses might benefit from faster examination times. Second, patients with malignant breast tumors larger than 5 cm who undergo neoadjuvant chemotherapy may benefit from the automated technique because the handheld device has a smaller footprint (5 cm) and therefore is limited in the evaluation of extent of disease. Third, for patients with dense breast tissue impeding mammography, screening automated ultrasound may be beneficial because of improved workflow efficiency and lack of operator dependence. Fourth, for surgical planning, surgeons who are familiar with the coronal plane may appreciate the multiplanar images obtained with automated ultrasound. Last, second-look ultrasound after breast MRI can be avoided because the 3D volume data on the whole-breast parenchyma can be reviewed after breast MRI. Our study had several weaknesses. First, the readers were blinded to the mammographic and clinical findings, and BI-RADS categorization established only with the ultrasound features does not always reflect actual practice. For example, some lesions might have been classified as category 4 at mammography but as category 3 at ultrasound, and biopsy would have been recommended on the basis of the mammographic finding despite the ultrasound evaluation. Second, our study included a relatively small number of patients. In addition, the number of special cases, such as calcifications and lymph nodes, was relatively small, and there was a lack of clustered microcysts. Third, some descriptors, such as intraductal mass and clustered microcyst, were not assessed. These descriptors should be prospectively evaluated for correlation with the mammographic and clinical findings. Allowing the readers to choose more than one descriptor for the margins, because more than one contour characteristic can be present in the same mass, would have been more realistic. Fourth, none of the 36 lesions detected only with automated ultrasound was biopsied or followed long enough to confirm the benign histologic finding. Further study is needed to confirm our results. Fifth, we did not compare all findings of handheld and automated ultrasound. We compared the findings of the two techniques only for lesion location and final assessment because not all readers reevaluated the captured handheld ultrasound image. Instead, one radiologist matched the findings of handheld ultrasound and automated ultrasound across five readers. Conclusion With a standardized scanning protocol and interpretation criteria for automated ultrasound, we found that detection of lesions larger than 1.2 cm in largest diameter was reliable. High reliability was obtained for reporting lesion size and location, but there was a relatively small number of special cases such as calcifications and lymph nodes and a lack of clustered microcysts. Further studies with a larger number of patients are needed to confirm these results. References 1. Bassett LW. Imaging of breast masses. Radiol Clin North Am 2000; 38: AJR:197, September

8 Shin et al. 2. Berg WA, Gutierrez L, NessAiver MS, et al. Diagnostic accuracy of mammography, clinical examination, US, and MR imaging in preoperative assessment of breast cancer. Radiology 2004; 233: Helbich TH, Matzek W, Fuchsjäger MH. Stereotactic and ultrasound-guided breast biopsy. Eur Radiol 2004; 14: Shapiro S, Venet W, Strax P, Venet L, Roeser R. Tento fourteen-year effect of screening on breast cancer mortality. J Natl Cancer Inst 1982; 69: Mandelson MT, Oestreicher N, Porter PL, et al. Breast density as a predictor of mammographic detection: comparison of interval- and screen-detected cancers. J Natl Cancer Inst 2000; 92: Boyd NF, Guo H, Martin LJ, et al. Mammographic density and the risk and detection of breast cancer. N Engl J Med 2007; 356: Mendelson EB, Tobin CE. Critical pathways in using breast US. RadioGraphics 1995; 15: Flobbe K, Bosch AM, Kessels AG, et al. The additional diagnostic value of ultrasonography in the diagnosis of breast cancer. Arch Intern Med 2003; 163: Houssami N, Irwig L, Simpson JM, McKessar M, Blome S, Noakes J. Sydney Breast Imaging Accuracy Study: comparative sensitivity and specificity of mammography and sonography in young women with symptoms. AJR 2003; 180: Berg WA, Gilbreath PL. Multicentric and multifocal cancer: whole-breast US in preoperative evaluation. Radiology 2000; 214: Hlawatsch A, Teifke A, Schmidt M, Thelen M. Preoperative assessment of breast cancer: sonography versus MR imaging. AJR 2002; 179: Moon WK, Noh DY, Im JG. Multifocal, multicentric, and contralateral breast cancers: bilateral whole-breast US in the preoperative evaluation of patients. Radiology 2002; 224: Gordon PB, Goldenberg SL. Malignant breast masses detected only by ultrasound: a retrospective review. Cancer 1995; 76: Kaplan SS. Clinical utility of bilateral wholebreast US in the evaluation of women with dense breast tissue. Radiology 2001; 221: Kolb TM, Lichy J, Newhouse JH. Comparison of the performance of screening mammography, physical examination, and breast US and evaluation of factors that influence them: an analysis of 27,825 patient evaluations. Radiology 2002; 225: Crystal P, Strano SD, Shcharynski S, Koretz MJ. Using sonography to screen women with mammographically dense breasts. AJR 2003; 181: Leconte I, Feger C, Galant C, et al. Mammography and subsequent whole-breast sonography of nonpalpable breast cancers: the importance of radiologic breast density. AJR 2003; 180: Buchberger W, Niehoff A, Obrist P, DeKoekkoek- Doll P, Dünser M. Clinically and mammographically occult breast lesions: detection and classification with high-resolution sonography. Semin Ultrasound CT MR 2000; 21: Kopans DB. Breast-cancer screening with ultrasonography. Lancet 1999; 354: Berg WA, Blume JD, Cormack JB, et al. Combined screening with ultrasound and mammography vs mammography alone in women at elevated risk of breast cancer. JAMA 2008; 299: Corsetti V, Ferrari A, Ghirardi M, et al. Role of ultrasonography in detecting mammographically occult breast carcinoma in women with dense breasts. Radiol Med 2006; 111: Chou YH, Tiu CM, Chiang HR, et al. Ultrasound BI-RADS categories applied in an automated breast ultrasound system: diagnostic reliability (abstr). RSNA Oak Brook, IL: Radiological Society of North America, rsna.org/index.cfm?em_id= &event=1. Accessed May 17, Destounis S, Young W, Murphy P, et al. Initial experience of automated breast ultrasound screening trial in the setting of a community based private practice (abstr). RSNA Oak Brook, IL: Radiological Society of North America, id= &event=1. Accessed May 17, Wenkel E, Heckmann M, Heinrich M, et al. Automated breast ultrasound: lesion detection and BI- RADS classification a pilot study. Rofo 2008; 180: Kelly KM, Dean J, Comulada WS, Lee SJ. Breast cancer detection using automated whole breast ultrasound and mammography in radiographically dense breasts. Eur Radiol 2010; 20: Kelly KM, Dean J, Lee SJ, Comulada WS. Breast cancer detection: radiologists performance using mammography with and without automated wholebreast ultrasound. Eur Radiol 2010; 20: Mendelson EB, Baum JK, Berg WA, et al. BI- RADS: ultrasound, 1st ed. In: D Orsi CJ, Mendelson EB, Ikeda DM, et al. Breast Imaging Reporting and Data System: ACR BI-RADS breast imaging atlas. Reston, VA: American College of Radiology, Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977; 33: Skaane P, Engedal K, Skjennald A. Interobserver variation in the interpretation of breast imaging: comparison of mammography, ultrasonography, and both combined in the interpretation of palpable noncalcified breast masses. Acta Radiol 1997; 38: Berg WA, Campassi C, Langenberg P, Sexton MJ. Breast Imaging Reporting and Data System: interand intraobserver variability in feature analysis and final assessment. AJR 2000; 174: Baker JA, Kornguth PJ, Soo MS, Walsh R, Mengoni P. Sonography of solid breast lesions: observer variability of lesion description and assessment. AJR 1999; 172: Berg WA, Blume JD, Cormack JB, Mendelson EB. Operator dependence of physician-performed whole-breast US: lesion detection and characterization. Radiology 2006; 241: Berg WA, Blume JD, Cormack JB, Mendelson EB, Madsen EL; ACRIN 6666 Investigators. Lesion detection and characterization in a breast US phantom: results of the ACRIN 6666 investigators. Radiology 2006; 239: Abdullah N, Mesurolle B, El-Khoury M, Kao E. Breast imaging reporting and data system lexicon for US: interobserver agreement for assessment of breast masses. Radiology 2009; 252: Berg WA. Supplemental screening sonography in dense breasts. Radiol Clin North Am 2004; 42: AJR:197, September 2011