Interobserver Variability of Ultrasound Elastography: How It Affects the Diagnosis of Breast Lesions

Women s Imaging Original Research Yoon et al. Elastography of Breast Lesions Women s Imaging Original Research Downloaded from www.ajronline.org by 46.3.21.182 on 1/24/18 from IP address 46.3.21.182. Copyright ARRS. For personal use only; all rights reserved Jung Hyun Yoon 1 Myung Hyun Kim Eun-Kyung Kim Hee Jung Moon Jin Young Kwak Min Jung Kim Yoon JH, Kim MH, Kim EK, Moon HJ, Kwak JY, Kim MJ 1 All authors: Department of Radiology, Research Institute of Radiological Science, Yonsei University, College of Medicine, 25 Seongsanno, Seodaemun-gu, Shinchondong, Seoul 12-752, Republic of Korea. Address correspondence to M. J. Kim (mines@yuhs.ac). AJR 211; 196:73 736 WOMEN S IMAGING Keywords: BI-RADS, breast, elastography, interobserver variability, ultrasound DOI:1.2214/AJR.1.4654 Received March 21, 21; accepted after revision July 24, 21. 361 83X/11/1963 73 American Roentgen Ray Society Interobserver Variability of Elastography: How It Affects the Diagnosis of Breast Lesions Objective. The purpose of this article is to evaluate the interobserver variability of elastography on real-time ultrasound and how it influences the agreement of final assessment on ultrasound. Subjects and Methods. From April to May 29, 65 breast lesions of 53 patients (mean age, 42.6 years; range, 18 72 years) who underwent ultrasound-guided core biopsy were included in this study. and elastography images of the lesion subjected to biopsy were obtained and prospectively analyzed by three radiologists with individual realtime image scanning prior to biopsy. Each radiologist recorded final ultrasound BI-RADS assessments using ultrasound and combined ultrasound and elastography and the fat-to-lesion ratio and elasticity score. The histopathologic results obtained from ultrasound-guided core biopsy or excision were used as the reference standard. Diagnostic performances and interobserver agreement were analyzed. RESULTS. Of the 65 lesions, 43 (66.2%) were benign, and 22 (33.8%) were malignant. Specificity (2.2 33.3%), positive predictive value (38.7 45.1%), and accuracy (46.7 55.4%) were significantly improved in combined ultrasound and elastography (p <.1). Area under the curve (AUC) values for all three performers did not show significant differences in ultrasound (AUC,.959) and combined ultrasound and elastography (AUC,.957) (p =.92). Interobserver agreement was not improved with combined ultrasound and elastography (κ =.25) in comparison to ultrasound only (κ =.37). Interobserver agreement of real-time elastography was fair in both fat-to-lesion ratio (intraclass correlation coefficient score,.25) and elasticity score (κ =.28). Moderate agreement (κ =.46) was seen with static elastography. Conclusion. Elastography improves the specificity, positive predictive value, and accuracy of ultrasound. However, significant interobserver variability exists, with real-time elastographic performance showing fair agreement. E lastography is an imaging tool that may reliably augment mammography and ultrasound with the aim of improving diagnostic accuracy noninvasively. Elastography produces images that map the strain in tissue elements subjected to external compression [1, 2]. This technique depends mainly on the fact that cancer tissue is harder than the surrounding normal parenchyma [3] and that harder tissue is more resistant to compression than softer tissues. Tissue hardness can be estimated, which is another characteristic of morphologic features seen on ultrasound that may be used in differential diagnosis of variable breast lesions. Harder areas show less tissue displacement, displaying darker strain images, whereas softer areas show more displacement and brighter strain images [3, 4]. This analysis is based on the assumption that the tissue displacement occurs in the longitudinal direction, which is the same direction to the beam. However, the breast is a curved structure with redundancy, which makes it difficult to apply homogeneous continuous compression motions on the breast lesion, especially when lesions vary in location within the breast. Varying compression techniques among performers may also influence the strain images, resulting in interobserver variability in elastography interpretation of the same breast lesion [3]. Considering these factors, elastography has major limitations in that it is a subjective imaging method. Several studies have been published on the inter- or intraobserver variability of elastography conducted on breast lesions [5, 6], but those studies were based on the static or recorded motion images obtained by one 73 AJR:196, March 211

Elastography of Breast Lesions Downloaded from www.ajronline.org by 46.3.21.182 on 1/24/18 from IP address 46.3.21.182. Copyright ARRS. For personal use only; all rights reserved performer. To our knowledge, studies of image reproducibility and intra- or interobserver variability of elastography in real-time evaluation of the breast have not yet been published. Thus, the purpose of our study was to evaluate the interobserver variability of elastography in the diagnosis of breast lesions on real-time ultrasound and how elastography influences the agreement of final assessment on conventional ultrasound. We also evaluated the interobserver variability of static elastography images in comparison with the real-time elastography assessments. Subjects and Methods Patients This study was conducted with institutional review board approval, and signed informed consent was obtained from all patients before ultrasound evaluation and biopsy. From April to May 29, 65 consecutive breast lesions in the 53 women referred for ultrasound-guided core needle biopsy on the basis of imaging findings or physician s decision were included in this study. All 65 breast lesions underwent ultrasound, elastography, and subsequent ultrasound-guided core needle biopsy. The age of the patients ranged from 18 to 72 years (mean, 42.6 years). Of the lesions included, 34 (52.3%) were palpable and 31 (47.7%) were asymptomatic. Examinations and Biopsy Procedures and elastography images were obtained by using a 6 14-MHz linear array transducer (EUB-75, Hitachi Medical). Real-time whole breast ultrasound examination was performed by one of three board-certified radiologists, with different levels of experience in breast imaging (range, 1 7 years). The three radiologists involved in this study practiced obtaining optimal elastography images for 3 months on patients with focal breast lesions during daily examinations. After the whole breast ultrasound examination, ultrasound and elastography were performed sequentially only at the lesion subjected to biopsy by the three radiologists in turn. All three radiologists prospectively analyzed the ultrasound images and recorded the assessments of ultrasound alone and ultrasound after elastography, hereafter referred to as combined ultrasound and elastography. Final assessments of ultrasound and combined ultrasound and elastography were made according to the American College of Radiology s BI-RADS [7]. Elastography scores from elastography images were also individually recorded by the three performers. Performers were blinded to the results of others. After all three radiologists were finished, biopsy was performed by the first radiologist who performed the whole breast ultrasound examination. Local anesthesia (1% lidocaine) was routinely applied, and an automated gun (Pro-Mag 2.2, Manan Medical Products) and a 14-gauge Tru-Cut needle with a 22- mm throw (SACN Biopsy Needle, Medical Device Technologies) were used. Informed patient consent was obtained for all biopsy procedures. Real-Time Elastography elastography was performed using a freehand technique at the same time as ultrasound. Images were obtained by applying repetitive light compression at the skin above the targeted breast lesion. The probe was positioned perpendicular to the skin when applying pressure. The ultrasound scanner was equipped with an elastography unit, images were presented in a split-screen mode with the conventional images in the right, and the translucent color-scale elastography images were superimposed on the corresponding ultrasound image in the left. A square region of interest (ROI) was set for elastography acquisition; the superior margin was set to include subcutaneous fat, the inferior margin was set to include pectoral muscle, and the lateral margin was set to include more than 5 mm of breast parenchyma adjacent to the targeted lesion. Light repetitive compression motions were applied to the lesion using the ultrasound probe, and for optimal elastography acquisition, performers obtained images showing either color homogeneity within the ROI or the pressure indicator displayed on the screen, ranging between numbers of 2 and 3. Each pixel of the elasticity image was shown as one of the 256 specific colors, representing the extent of strain. The scale ranged from red, showing areas of greatest strain (i.e., softest component), to blue, showing no strain (i.e., hardest component). The fat-to-lesion ratio and elasticity score were obtained from the elastography images. The fat-to-lesion ratio, also described as the strain index [8], was automatically calculated by the ultrasound scanner according to the fat-tomass strain ratio [8]. The fat-to-lesion ratio was obtained by applying two additional ROIs to the superimposed elastography images. A sample image used to obtain the fat-to-lesion ratio is shown in Figure 1. A round or oval ROI was applied to the area showing homogeneously green elasticity (i.e., average strain) among the subcutaneous fat layers. A round, oval, or polygonal ROI was applied at the targeted lesion on the B-mode image for maximum inclusion in the elasticity measurement. The elasticity score was classified individually by each performer, on a scale of 1 to 5, as proposed by Itoh et al. [3]. A score of 1 indicated even strain in the entire hypoechoic lesion. A score of 2 indicated even strain throughout the lesion, with some strain-free areas (i.e., elastography showing mosaic pattern of green and blue). A score of 3 indicated strain only in the periphery of the lesion, not in the center. A score of 4 indicated no strain in the entire lesion. A score of 5 indicated no strain within the lesion and also the surrounding areas of the hypoechoic lesion. BI-RADS assessments of ultrasound, elastography scores, fatto-lesion ratios, and final BI-RADS assessments after elastography were recorded on a prepared sheet by each performer. Static Elastography After the initial evaluation, each elastography image obtained by the three performers was selected for all the included breast lesions. A total of 195 elastography images of the 65 breast lesions were retrospectively reviewed by the performers during a single review session. Images were converted into JPEG files and randomly displayed during one image review session. All three performers were blinded to the patient s demographics and to which image was obtained by whom during review. Elasticity scores were graded individually by each performer according to the previously mentioned scale Fig. 1 Measurement of fat-to-lesion ratio in real-time elastography images. Round, oval, or polygonal region of interest (ROI) (A, right panel) was applied at targeted lesion for maximum inclusion in elasticity measurement. Round or oval ROI (B, left panel) was applied to area showing homogeneously green elasticity among subcutaneous fat layers. Fat-to-lesion ratio was automatically calculated by the ultrasound scanner. AJR:196, March 211 731

Yoon et al. Downloaded from www.ajronline.org by 46.3.21.182 on 1/24/18 from IP address 46.3.21.182. Copyright ARRS. For personal use only; all rights reserved TABLE 1: Histologic Diagnosis of Benign and Malignant Breast Lesions in 65 Patients No. of Histopathologic Diagnosis Lesions Benign lesions (n = 43) Fibroadenoma 1 Fibroadenomatoid hyperplasia 15 Fibrocystic disease 4 Phyllodes tumor, benign 1 Mastitis, chronic or granulomatous 1 Papilloma 2 Fibrosis 4 Benign cyst, galactocele 2 Schwannoma, benign 1 Ductal hyperplasia 3 Malignant lesions (n = 22) Invasive ductal carcinoma 15 a Ductal carcinoma in situ 1 Metastasis 3 Tubular carcinoma 1 Papillary carcinoma 1 Phyllodes tumor, malignant 1 b Total 65 a Two cases were invasive ductal carcinoma with ductal carcinoma in situ. b Fibroadenoma on core biopsy was upgraded into malignant phyllodes tumor on surgical excision. of 1 5. All three performers were unaware of the other performers scores. Results were recorded on a prepared file for analysis by each performer. Data and Statistical Analysis For statistical analysis, final assessments based on ultrasound BI-RADS were grouped into two groups; positive assessments consisted of categories 4a, 4b, 4c, and 5, and negative assessments consisted of categories 2 and 3. If final assessments of a breast lesion differed between assessments made with ultrasound alone and those made with combined ultrasound and elastography, the higher BI- RADS assessments were used as the final assessment for the patient. Elasticity score was grouped into two groups according to the study of Itoh et al. [3]; scores 4 and 5 were considered to be positive, and scores 1 3 were considered to be negative. Histopathologic results from the ultrasoundguided core needle biopsy were used as the reference standard. In patients diagnosed with atypia on core biopsy, results of subsequent surgical excision were used as the reference standard. Diagnostic indexes, including sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and accuracy, for ultrasound and combined ultrasound and elastography were calculated. McNemar s test and the generalized estimation equation method were used in the comparison of the diagnostic indexes. For overall performances of ultrasound and combined ultrasound and elastography, area under the receiver operating characteristic (ROC) curve (AUC) values were obtained and compared. Kappa statistics were calculated to assess the agreement of final BI-RADS assessment of ultrasound, combined elastography and ultrasound, and elasticity scores. We also evaluated kappa statistics when we divided the BI-RADS assessment and elasticity scores dichotomously as described previously in the article. Intraclass correlation coefficient scores were calculated for agreement of fatto-lesion ratio. Estimation of an overall kappa value is based on the study of Landis and Koch [9]. A kappa value of corresponds to no agreement, a kappa value of 1. corresponds to complete agreement, and a kappa value of less than corresponds to disagreement. Kappa values less than or equal to.2 indicate slight agreement, values of.21.4 indicate fair agreement, values of.41.6 indicate moderate agreement, values of.61.8 indicate substantial agreement, and values of.81 1. indicate almost perfect agreement. All statistical analyses in this study were performed with SAS software (version 9.1, SAS Institute), and p less than.5 was considered statistically significant. Results Among the 65 breast lesions, 43 (66.2%) were diagnosed as benign, and 22 (33.8%) were diagnosed as malignant on ultrasoundguided core needle biopsy. Histologic diagnoses of the 65 breast lesions are summarized in Table 1. Lesion size ranged from 4 to 41 mm in maximum diameter (mean diameter, 14.9 mm; median diameter, 13. mm; SD, 8. mm). One patient was diagnosed as having atypical epithelial hyperplasia on core needle biopsy and underwent surgical excision for further evaluation. Final pathologic analysis proved to be papillary lesions with florid ductal hyperplasia. Diagnostic Performances of, Combined and Elastography, and Elastography Diagnostic indexes for ultrasound and combined ultrasound and elastography are summarized in Table 2. For all three performers, specificity, PPV, and accuracy were significantly improved in combined ultrasound and elastography, from 2.2% to 33.3%, 38.7% to 45.1%, and 46.7% to 55.4%, respectively (p <.1). For all three performers, no significant differences were seen in sensitivity or NPV for the two imaging methods. ROC curves of ultrasound and combined ultrasound and elastography are displayed in Figure 2. Overall diagnostic performances for all three performers were excellent. Among the three performers, one showed slightly improved AUC value, whereas the remaining two performers showed decreased AUC values in combined ultrasound and elastography, without statistical significance (p >.5). Regarding the ROC curves of fat-to-lesion ratio in elastography and elasticity scores, AUC values were higher for fat-to-lesion ratio (AUC,.855) compared with elasticity score (AUC,.791) but were not significantly different (p =.8). Interobserver Agreements of and Combined and Elastography Results of the interobserver agreements are summarized in Table 3. Fair agreement (κ =.37) was seen among the three performers for BI-RADS assessment of ultrasound. Fair agreement with decreased kappa values (κ =.25) was seen for combined ultrasound and elastography. When the BI-RADS assessment was dichotomously divided, fair agreement was observed for both ultrasound and combined ultrasound and elastography, with slight improvement of kappa values (κ =.4 and.34, respectively). Representative cases are displayed in Figures 3 and 4. Interobserver Agreement for Real-Time and Static Elastography Fair agreement, with an intraclass correlation coefficient value of.25 (lower,.8; upper,.4), was seen among the three performers for fat-to-lesion ratio of elastography. Fair agreement (κ =.28) was observed for the agreement of elasticity scores. When elasticity scores were dichotomously divided, fair agreement was also observed in the two groups, with slightly increased kappa values (κ =.36). Moderate agreement (κ =.46) was seen in elasticity scores during random review of elastography images. When elasticity scores were dichotomously divided, substantial agreement was seen in the two groups, with increased kappa values (κ =.79). Discussion Elastography shows the relative degree of the tissue strain when external compression is 732 AJR:196, March 211

Elastography of Breast Lesions Downloaded from www.ajronline.org by 46.3.21.182 on 1/24/18 from IP address 46.3.21.182. Copyright ARRS. For personal use only; all rights reserved TABLE 2: Diagnostic Performances of UItrasound and Combined and Elastography Negative Predictive Value (%) Accuracy (%) Area Under the Curve Value Specificity (%) Positive Predictive Value (%) Elastography Elastography Elastography Elastography Elastography Elastography Performer 1 22/22 () 22/22 () 13/43 (3.2) 18/43 (41.9) 22/52 (42.3) 22/4 (55.) 13/13 () 22/22 () 35/65 (53.9) 4/65 (61.5).965.97 2 21/22 (95.5) 21/22 (95.5) 7/43 (16.3) 1/43 (23.3) 21/57 (36.8) 21/54 (38.9) 7/8 (87.5) 1/11 (9.9) 28/65 (43.1) 31/65 (47.7).931.925 3 22/22 () 22/22 () 6/43 (14.) 15/43 (34.9) 22/59 (37.3) 22/5 (44.) 6/6 () 15/15 () 28/65 (43.1) 37/65 (56.9).987.969 All 65/66 (98.5) 65/66 (98.5) 26/129 (2.2) 43/129 (33.3) a 65/168 (38.7) 65/144 (45.1) a 26/27 (96.3) 47/48 (97.9) 91/195 (46.7) 18/195 (55.4) a.959.957 a p <.5. 8 6 4 2 8 6 4 2 2 2 plus elastography p =.919 4 6 Specificity 8 plus elastography p =.343 4 6 Specificity 8 A 8 6 4 2 8 6 4 2 2 2 plus elastography p =.871 4 6 Specificity applied [1], providing additional but different information of the target lesion than ultrasound does. In our study, specificity, PPV, and accuracy were significantly improved with combined ultrasound and elastography, but NPV and AUC values did not show significant differences between the two groups; rather, AUC values decreased for two performers. As in our study, other recent studies proved that elastography improved the specificity and overall diagnostic value of ultrasound [11, 12], but AUC values did not show significant improvement with elastography [11, 13]. Because elastography is individually performed, usually by freehand technique, variability is inevitable because of the nonaxial compression motions of the performer and the diverse slopes of the breast in different patients and locations of the breast [14]. Also, even with the same elastographic image, variable interpretations are possible among performers because image selection is mostly made by the performer [3]. Many previous studies have described the interobserver variability as a limitation of elastography [4, 15], but to our knowledge, there are no previous studies that evaluated the actual variabilities between performers in real-time elastography. A recent study evaluated in vitro intraand interobserver validations among performers [16] while different scanning parameters were applied, showing fair to moderate interobserver agreement. However, that study was performed with a tissue-mimicking phantom, and imaging was performed under the precise given scanning parameter, which may have contributed to the results. Further studies with a clinical basis (i.e., in vivo) are essential for more reliable usage of elastography. Our results show that significant variability, showing fair agreement, exists in real-time elastography between performers. Various fac- 8 plus elastography p =.873 4 6 Specificity C Fig. 2 Receiver operating characteristic curves for ultrasound and combined ultrasound and elastography. A, Overall performance for all three performers (area under the curve [AUC] values for ultrasound and combined ultrasound and elastography,.959 and.957, respectively). B, Performer 2, with 1 year of experience (AUC values,.931 and.925). C, Performer 3, with 4 years of experience (AUC values,.987 and.969). D, Performer 1, with 7 years of experience (AUC values,.965 and.97). 8 B D AJR:196, March 211 733

Yoon et al. Downloaded from www.ajronline.org by 46.3.21.182 on 1/24/18 from IP address 46.3.21.182. Copyright ARRS. For personal use only; all rights reserved TABLE 3: Interobserver Agreement of, Combined and Elastography, and Elastography Agreement Combined and Elastography Fat-to-Lesion Ratio Elastography Score Static Elastography Score Overall.37.25.25.28.46 Dichotomous division: positive vs negative.4.34 Not calculated.36.79 Note Data are kappa values. tors that are known to affect elastography images include patient factors (e.g., breast size and density), lesion factors (e.g., size, location, and depth), acquisition process factors (e.g., the type of ultrasound elastography device, extent of tissue compression, and performer variability), and interpretation variability [3, 5]. Because the patient, target lesion, and ultrasound elastography device were equal for all performers, the variability in the present study may have been largely influenced by the differences in image acquisition or interpretation during the freehand procedure. A We reviewed the static elastography images to assess interobserver variability according to the interpretational differences in identical elastographic images. Although agreement for real-time elastography images was fair, regardless of the simplification of elasticity scores, moderate-to-substantial agreement was observed in review of the static elastography images. This finding suggests that the significant variability in realtime elastography may be related to the performance of real-time elastography that is, the varying compression motions or, in wider terms, inadequate data acquisition rather than inaccurate interpretation. Therefore, with standardization of image acquisition procedures, further improvement of performance may be achieved. As in our study, the relative color display of the background breast parenchyma or chest wall in the elastography images or a built-in pressure indicator has been used for optimal and uniform compression in few recent studies [17, 18]. However, there are no standardized criteria indicating adequate compression, implying that each performer s subjective experience B C Fig. 3 55-year-old woman with palpable lesion in left breast. A, Performer 1 regarded this lesion (arrows) as category 4c, performer 2 as category 4b, and performer 3 as category 4c on ultrasound. B D, Elastography scores for performers 1 (B), 2 (C), and 3 (D) were 4, 3, and 2, respectively. On combined ultrasound and elastography, final assessments for each performer were category 5, 4c, and 4b, respectively. Subsequent biopsy and surgery proved this lesion to be invasive ductal carcinoma. D 734 AJR:196, March 211

Elastography of Breast Lesions Downloaded from www.ajronline.org by 46.3.21.182 on 1/24/18 from IP address 46.3.21.182. Copyright ARRS. For personal use only; all rights reserved and probe operating skills directly affect elastographic results. Studies of standardizing compression of elastography imaging are needed in the future. Our results, showing fair agreement for both ultrasound and combined ultrasound and elastography, are consistent with the results of previous reports showing poor-to-moderate agreement (κ =.17.58) in assessing breast lesions according to BI-RADS category [19, 2]. We applied the subdivisions of category 4 in BI-RADS assessment of breast lesions, and, as in the previous studies, this may have lessened the agreement of ultrasound and combined ultrasound and elastography among performers in our study. Agreement among performers was only slightly improved when BI-RADS categories were divided dichotomously. Patients included in this study A C D Fig. 4 33-year-old woman with palpable lesion in right breast. A, Performer 1 regarded this lesion (arrows) as category 3, and performers 2 and 3 regard it as category 4a on ultrasound. B D, Elastography score of performers 1 (B), 2 (C), and 3 (D) were 3, 2, and 2, respectively. On combined ultrasound and elastography, final assessment for all performers was category 4a. -guided core needle biopsy proved this lesion to be benign fibroadenomatoid hyperplasia. were scheduled for biopsy for known breast lesions, which may have had an effect on the final assessment for some performers. In addition, information obtained during real-time performances could affect the interobserver agreement compared with the limited information of static elastography review. Most of the reports on interobserver agreement using ultrasound are based on static images, but during real-time examinations, the performers tend to consider many clinical factors, which may amplify the interobserver variability. This also explains why the agreement for ultrasound in our study was relatively low compared with that in previous reports [19]. When the elastography scores are divided into dichotomous groups in our study, interobserver agreement improved during both real-time and static analyses of elastography. Providing indications of lesions that may be classified as malignant and simplifying the elastography scale may help improve the agreement among performers, and further studies of interobserver agreement based on a simplified elastography scale that does not affect the diagnostic performances are needed in the future. The newly introduced parameter, fat-tolesion ratio, may appear more objective by the display of calculated numeric values from the ultrasound machine, but in our study this ratio did not show any significance in improving diagnostic performance or interobserver agreement. ROC curves of fat-to-lesion ratio in elastography and elastography scores showed good AUC values (.855 and.791, respectively) without significant differences (p =.8). Because the fat-to-lesion ratio requires man- B AJR:196, March 211 735

Yoon et al. Downloaded from www.ajronline.org by 46.3.21.182 on 1/24/18 from IP address 46.3.21.182. Copyright ARRS. For personal use only; all rights reserved ual tracing of the lesion, we do not consider that it offers any additional information compared with the elastography score. In addition, we did not evaluate whether the differences of fat-to-lesion ratio between the performers were from the interobserver variability during real-time elastography or from the interobserver variability during procedures of obtaining fat-to-lesion ratio. Our study has some limitations. First, patients included were mostly referred and were scheduled for biopsies of known breast lesions. Final assessments may have been overrated, especially for the performer involved with the biopsy procedures, which may have had the effect of lowering the specificity. Second, lesions included were variable in size, ranging from 4 to 41 mm. In large lesions, the ROI may not have sufficiently included the surrounding breast parenchyma in the measurement of fat-to-lesion ratio, leading to suboptimal results. On the contrary, elastography of small lesions measuring less than 5 mm may have shown suboptimal results and limited characterization. Although it has been proven that elasticity scores are significantly higher in malignant lesions regardless of tumor size [18], there may have been certain limitations in specifically characterizing and grading the elasticity scores in small lesions. Third, an elasticity score of 3 was regarded as benign on dichotomous division during statistical analysis. In the study of Itoh et al. [3], an elasticity score of 3 showed a low malignancy rate (13%), and lesions with an elasticity score of 3 were regarded as benign, in consideration of the lesser probability of it being malignant. Lesions with elasticity scores of 3 that proved to be malignant may have had an effect on the diagnostic performances of elastography. Fourth, each performer selected the most adequate image from the images that he or she performed, which may have induced selection bias. Also, radiologists involved in this study were relatively inexperienced, which may have affected the interobserver variability of ultrasound and elastography. Although they had limited experience, all three radiologists showed good performance in ultrasound with AUC values over.9; experience itself may not have had significant influence on interobserver variability. In conclusion, elastography may improve the specificity, PPV, and accuracy of ultrasound, but did not show significant improvement in AUC values of ultrasound. Significant interobserver variability exists, with real-time elastography showing fair agreement, despite the moderate-to-substantial agreement seen in the assessment of static elastography images. Further evaluation of the parameters affecting elastography may help reduce the interobserver variability in elastography. References 1. Ophir J, Cespedes I, Ponnekanti H, Yazdi Y, Li X. Elastography: a quantitative method for imaging the elasticity of biological tissues. Ultrason Imaging 1991; 13:111 134 2. Thitaikumar A, Ophir J. Effect of lesion boundary conditions on axial strain elastograms: a parametric study. Med Biol 27; 33:1463 1467 3. Itoh A, Ueno E, Tohno E, et al. Breast disease: clinical application of US elastography for diagnosis. Radiology 26; 239:341 35 4. Regner DM, Hesley GK, Hangiandreou NJ, et al. Breast lesions: evaluation with US strain imaging clinical experience of multiple observers. Radiology 26; 238:425 437 5. Thomas A, Fischer T, Frey H, et al. Real-time elastography: an advanced method of ultrasound first results in 18 patients with breast lesions. Obstet Gynecol 26; 28: 335 34 6. Schaefer FK, Heer I, Schaefer PJ, et al. Breast ultrasound elastography: results of 193 breast lesions in a prospective study with histopathologic correlation. Eur J Radiol 29 Sep 19 [Epub ahead of print] 7. American College of Radiology. Breast imaging reporting and data system (BI-RADS), 4th ed. Reston, VA: American College of Radiology, 23 8. Cho N, Moon WK, Kim HY, Chang JM, Park SH, Lyou CY. Sonoelastographic strain index for differentiation of benign and malignant nonpalpable breast masses. J Med 21; 29:1 7 9. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977; 33:159 174 1. Tohno E, Ueno E. Current improvements in breast ultrasound, with a special focus on elastography. Breast Cancer 28; 15:2 24 11. Cho N, Moon WK, Park JS, Cha JH, Jang M, Seong MH. Nonpalpable breast masses: evaluation by US elastography. Korean J Radiol 28; 9:111 118 12. Sohn YM, Kim MJ, Kim EK, Kwak JY, Moon HJ, Kim SJ. Sonographic elastography combined with conventional sonography: how much is it helpful for diagnostic performance? J Med 29; 28:413 42 13. Scaperrotta G, Ferranti C, Costa C, et al. Role of sonoelastography in non-palpable breast lesions. Eur Radiol 28; 18:2381 2389 14. Hiltawsky KM, Kruger M, Starke C, Heuser L, Ermert H, Jensen A. Freehand ultrasound elastography of breast lesions: clinical results. Med Biol 21; 27:1461 1469 15. Fleury Ede F, Fleury JC, Piato S, Roveda D Jr. New elastographic classification of breast lesions during and after compression. Diagn Interv Radiol 29; 15:96 13 16. Havre RF, Elde E, Gilja OH, et al. Freehand realtime elastography: impact of scanning parameters on image quality and in vitro intra- and interobserver validations. Med Biol 28; 34:1638 165 17. Cho N, Moon WK, Park JS. Real-time US elastography in the differentiation of suspicious microcalcifications on mammography. Eur Radiol 29; 19:1621 1628 18. Zhu QL, Jiang YX, Liu JB, et al. Real-time ultrasound elastography: its potential role in assessment of breast lesions. Med Biol 28; 34:1232 1238 19. Lee HJ, Kim EK, Kim MJ, et al. Observer variability of Breast Imaging Reporting and Data System (BI-RADS) for breast ultrasound. Eur J Radiol 28; 65:293 298 2. Abdullah N, Mesurolle B, El-Khoury M, Kao E. Breast imaging reporting and data system lexicon for US: interobserver agreement for assessment of breast masses. Radiology 29; 252:665 672 736 AJR:196, March 211