Follow-up and Final Results of the Oslo I Study Comparing Screen-Film Mammography and Full-field Digital Mammography with Soft-Copy Reading

Transcription

1 Acta Radiologica ISSN: (Print) (Online) Journal homepage: Follow-up and Final Results of the Oslo I Study Comparing Screen-Film Mammography and Full-field Digital Mammography with Soft-Copy Reading P. Skaane, A. Skjennald, K. Young, E. Egge, I. Jebsen, E. M. Sager, B. Scheel, E. SØvik, A. K. Ertzaas, S. Hofvind & M. Abdelnoor To cite this article: P. Skaane, A. Skjennald, K. Young, E. Egge, I. Jebsen, E. M. Sager, B. Scheel, E. SØvik, A. K. Ertzaas, S. Hofvind & M. Abdelnoor (2005) Follow-up and Final Results of the Oslo I Study Comparing Screen-Film Mammography and Full-field Digital Mammography with Soft-Copy Reading, Acta Radiologica, 46:7, To link to this article: Published online: 09 Jul Submit your article to this journal Article views: 88 View related articles Citing articles: 3 View citing articles Full Terms & Conditions of access and use can be found at Download by: [ ] Date: 19 November 2017, At: 14:04

2 ORIGINAL ARTICLE ACTA RADIOLOGICA Follow-up and Final Results of the Oslo I Study Comparing Screen-Film Mammography and Full-field Digital Mammography with Soft-Copy Reading P. SKAANE, A.SKJENNALD, K.YOUNG, E.EGGE, I.JEBSEN, E.M.SAGER, B.SCHEEL, E.SØVIK, A. K. ERTZAAS, S.HOFVIND &M.ABDELNOOR Department of Radiology, Ullevaal University Hospital, Oslo, Norway; Volvat Medical Center, Oslo, Norway; Centrum Roentgen Institute, Oslo, Norway; Institute of Population-based Cancer Research, The Cancer Registry of Norway, Oslo, Norway; and Center for Clinical Research, Section of Epidemiology and Biostatistics, Ullevaal University Hospital, Oslo, Norway Downloaded by [ ] at 14:04 19 November 2017 Skaane P, Skjennald A, Young K, Egge E, Jebsen I, Sager EM, Scheel B, Søvik E, Ertzaas AK, Hofvind S, Abdelnoor M. Follow-up and final results of the oslo I study comparing screen-film mammography and full-field digital mammography with softcopy reading. Acta Radiol 2005;46: Purpose: To compare cancer detection rates of screen-film (SFM) and full-field digital mammography (FFDM) with soft-copy reading in a screening program including the initial positive scores for interval cancers and cancers in the subsequent screening round, and to analyze the false-negative FFDM interpretations. Material and Methods: Using a paired study design, 3683 women underwent SFM and FFDM in a population-based screening program. Two standard views of each breast were acquired. The images were interpreted without previous films for comparison. Independent double reading using a 5-point rating scale for probability of cancer was used for each modality. An examination was defined as positive if at least one of the two independent readers scored 2 or higher on the 5-point rating scale. SFM-positive cases were discussed in a SFM consensus meeting and FFDM-positive cases in a separate FFDM consensus meeting before recall. The study population was followed for more than 2 years so that interval cancers and screen-detected cancers in the subsequent screening round could be included. Cancer detection rates were compared using the McNemar test for paired proportions. The kappa statistic and Wilcoxon signed-rank test for matched pairs were used for comparing rating scores. The reading time was recorded for all FFDM interpretations. Results: A total of 31 cancers (detection rate 0.84%) were diagnosed initially, of which SFM detected 28 and FFDM 23 (McNemar test P50.23, discordant pair 8 and 3). Two cancers with a positive score at initial SFM reading and three with a positive score at initial FFDM reading were dismissed at SFM and FFDM consensus meetings, respectively. The difference in cancer detection after recall (discordant pair 11 and 5) was not significant (McNemar test, P50.21). Of the 10 interval cancers and 16 screendetected cancers in the subsequent round, 3 had true-positive SFM scores while 4 had true-positive FFDM scores in the initial reading session. A total of 38 cancers therefore had a positive result at double reading at one or both modalities, 31 at SFM and 27 at FFDM (McNemar test, P50.48). Comparison of SFM and FFDM interpretations using the mean score for each case revealed no statistically significant difference between the two modalities (Wilcoxon signed-rank test for matched pairs; P-value50.228). Two initial round cancers (one tumor found incidentally at work-up for a mass proved to be a simple cyst with a positive score at FFDM but a negative score at SFM, and one tumor with positive score at SFM but negative score at FFDM due to positioning failure) were excluded from the further analysis. Excluding these two cancers from comparison, there were 31% (22 of 72) false-negative SFM and 47% (34 of 72) false-negative FFDM individual interpretations. The overall mean interpretation time for normal FFDM examinations was 45 s. For most false-negative FFDM results, the reading time was shorter or longer than for normal examinations. The recorded FFDM interpretation time was noticeably short for several overlooked cancers manifesting as microcalcifications (ductal carcinoma in situ). Conclusion: There is no statistically significant difference in cancer detection rate between SFM and FFDM with soft-copy reading in a mammography screening program. DOI / # 2005 Taylor & Francis

3 Final Results of the Oslo I Study 680 Analysis of cancers missed at FFDM with soft-copy reading indicates that close attention has to be paid to systematic use of image display protocols. Key words: Breast neoplasms; cancer screening; comparative breast studies; digital mammography; interobserver variation; interval cancer Per Skaane, M.D., Ullevaal University Hospital, Department of Radiology, Breast Imaging Center, Kirkeveien 166, NO-0407 Oslo, Norway (fax , . per.skaane@ulleval.no) Accepted for publication 23 June 2005 Downloaded by [ ] at 14:04 19 November 2017 Full-field digital mammography (FFDM) offers several potential benefits in breast cancer screening programs, including simplified archival and transmission of images, elimination of technically unsatisfactory images, enhanced cancer detection especially in dense breast parenchyma because of increased contrast resolution, development of telemammography, and implementation of computer-aided detection. The benefits of digital technology are probably best realized with soft-copy reading. Three large-scale studies comparing analog screen-film mammography (SFM) and full-field digital mammography (FFDM) with soft-copy reading in asymptomatic women have been published (12, 16, 17). An extension of the first study was published a year later (11). In the Colorado Massachusetts trial (11, 12) and the initial results of the Oslo I study (17), a lower cancer detection rate was found on FFDM with soft-copy reading as compared with SFM, although the difference was not statistically significant. However, a higher cancer detection rate for FFDM was found in the Oslo II study, approaching statistical significance for the age group 50 to 69 years (16). The use of FFDM with soft-copy reading resulted in a significantly lower recall rate in the Colorado Massachusetts trial (12). Unlike the lower recall rate on FFDM in the study by LEWIN et al., however, the recall rate in both the Oslo I and II studies was higher for FFDM (12, 16, 17). There are important differences between the three reported studies using FFDM with soft-copy reading in a screening population. There was no double reading in the Colorado Massachusetts trial, although SFM and FFDM images were interpreted independently, and recall was high for both FFDM (11.8%) and SFM (14.9%) (12). Independent double reading was used for both the Oslo I and II studies, with recall rates at SFM of 3.5% and 2.5%, respectively, and at FFDM of 4.6% and 3.8%, respectively (16, 17). Double reading may result in a detection of additional cancers compared to single reading, and double reading with consensus or arbitration, as applied in the Oslo studies, may achieve a higher cancer detection rate as well as a reduction in the number of women recalled for diagnostic work-up (4). However, recall by consensus or arbitration may have a variable impact on recall rates and consequently the cancer detection rate depends on the policy used. Our first report on the Oslo I study was based only on the positive interpretation results with initial diagnostic work-up (17). Objectives of this study were to include all interval cancers and cancers in the subsequent screening round and to analyze the false-negative FFDM soft-copy interpretations in the Oslo I study. Material and Methods Study population The Oslo I study started on 3 January and ended on 22 June Of the 9932 women aged between 50 and 69 years who attended the Norwegian Breast Cancer Screening Program (NBCSP) during this period, only a limited number could be offered participation in the study because of limited capacity of the technologists. A total of 3683 women (37% of the attenders aged between 50 and 69 years) gave their consent to take part in the study. All 3683 women were informed about the study beforehand, and their participation in the project was voluntary. Each woman enrolled in the study signed a written consent form specifying that the data and images related to screening could be used for future research and scientific purposes. Since the women were invited to the official NBCSP, the screen-film mammography (SFM) was always performed first in case the woman wanted to withdraw from participating in the double examination. The study was approved by the Regional Ethics Committee. Imaging The 3683 women (mean age 58.2 years) representing the study population underwent both SFM and FFDM. The SFM examinations were all acquired on

4 681 P. Skaane et al. one of two Mammomat 300 (Siemens Medical Systems, Erlangen, Germany) with Kodak Min-R 2000 film and Min-R 2190 screens (Eastman Kodak, Rochester, N.Y., USA), standard and large formats. Molybdenum-molybdenum and 29 kv were always used. FFDM images were acquired on a Senographe 2000D (G.E. Healthcare, Buc, France) equipped with an automatic mode (automatic optimization of parameters, AOP) in which anode-filter combination and kv is selected automatically after analysis of a short pre-exposure. The AOP was used in accordance with the manufacturer s recommendations. The area of the image detector was cm. Mammograms for both imaging modalities (SFM as well as FFDM) included the two standard views craniocaudal (CC) and mediolateral oblique (MLO) of each breast. finding), the case was automatically selected for the consensus meeting. There was no time limit for the image interpretation. There were separate consensus meetings for positive SFM and FFDM examinations twice weekly. If available, mammograms from the previous screening rounds were offered at both SFM and FFDM consensus meetings. Each team was responsible for its own consensus meeting and the participants could dismiss cases with abnormal mammographic findings (positive scores). The women then went back to the screening program and would be invited to the NBCSP 2 years later. In our screening program, short-term follow-up is never recommended. A flow chart showing the study design of the Oslo I study was presented in the initial report (17). Downloaded by [ ] at 14:04 19 November 2017 Image interpretation Eight radiologists with more than 4 years of experience with screening mammography participated in the study (batch reading). Each radiologist interpreted 100 FFDM examinations using softcopy reading (test cases) before the project started. The eight readers were divided into two teams: one team interpreted SFM and the other team FFDM for 1 week. The teams alternated weekly between SFM and FFDM interpretations. The SFM as well as the FFDM were interpreted independently by two radiologists. Previous mammograms were not offered for interpretation sessions either for SFM or for FFDM. The FFDM examinations were interpreted using soft-copy reading on the G.E. review workstation, which included two high-resolution 2 K62.5 K monitors and a dedicated keypad. The recommended display protocol for FFDM reading included three steps: 1) All four images presented: two CC views back-to-back on one monitor and two MLO views back-to-back on the other; 2) both CC views, one on each monitor ( full size ); 3) both MLO views, one on each monitor ( full size ). This protocol, as well as further post-processing (windowing-level adjustments, zooming, inversion) of the images, was optional. Post-processing, however, was strongly recommended for all cases and at least on the two MLO full size views since, in general, most of the breast parenchyma is shown on these images. A 5-point rating scale for probability of cancer was used for both SFM and FFDM: 15normal/ definitely benign; 25probably benign; 35indeterminate finding; 45probably malignant; 55malignant. If at least one of the two readers categorized the mammographic finding as 2 or higher ( positive Diagnostic work-up Diagnostic work-up of women recalled was carried out within 2 weeks after the consensus meeting. The imaging work-up included spot-compression and magnification views, ultrasound, and magnetic resonance imaging (MRI) if needed. Fine-needle aspiration cytology was the standard technique for needle biopsies. All cytologic and histologic examinations were carried out at the Department of Pathology. All cancers in Norway are reported to the Norwegian Cancer Registry, and this cancer file is linked to that of the NBCSP. Thus, a 100% surveillance of patients included in our mammography screening program is possible. Interpretation time registration The reading times were recorded only for the FFDM examinations, since we were not allowed to make any changes in the database of the NBCSP (including only the SFM examinations) for the digital Oslo I project. For each case, time from the bar code registration (using the light pen) to confirming the result with the light pen (or a mouse-click) was recorded, i.e. the interpretation times were recorded indirectly. Since the majority of screening examinations are normal, this option is given automatically on the display so that the reader can confirm the normal score with a mouse click (or the light pen). Follow-up The interpretation results, results from diagnostic work-up of recalled patients, and the results from histology from patients undergoing surgery are all put into the database of the NBCSP, which is

5 Downloaded by [ ] at 14:04 19 November 2017 located at the Cancer Registry of Norway. This database is linked to the mammography screening database for each county, thus enabling absolute surveillance of all women included in the screening program. The study population was followed for more than 2 years to include all interval cancers within 2 years and cancers in the subsequent screening round. To ensure that all cancers in the study population had been registered in the database, a last printout was made 2.5 years after closing of the study. The initial mammographic examinations of the women with interval cancers and subsequent screening round cancers were re-evaluated and compared with the corresponding interpretations for SFM and FFDM of the initial reading sessions. If a rating score of 2 or higher had been given to the examination in the initial reading session and the cancer was visible in retrospect (either missed cancer or minimal sign lesion), the result was retrospectively considered as true positive unless another lesion in the same breast could have been the explanation for the positive rating score (our registration is sidebased, i.e. separate scores for the right and left breast). Statistical analysis The cancer detection rates for SFM and FFDM were calculated based on the true-positive scores in the initial interpretation session; as well as including the initial true-positive scores for interval cancers and cancers in the subsequent screening round. A rating score of 2 or higher on the 5-point rating scale for probability of cancer by at least one of the two independent readers was defined as a truepositive finding. The cancer detection rates for SFM and FFDM were compared using McNemar s test for paired proportions. A P-valuev0.05 was considered statistically significant. The kappa statistic was used for calculating the interobserver agreement on SFM and FFDM interpretation. The Wilcoxon signed-rank test for matched pairs was used to compare the independent double reading for cancers detected on one or both modalities. Results Of the 3,683 women in the study population who underwent both SFM and FFDM, SFM showed a positive interpretation in 442 cases and FFDM in 612 cases; 314 (71%) of the 442 SFM and 444 (73%) of the 612 FFDM cases with a positive score in the reading sessions were dismissed at consensus meetings. The recall rate was 3.5% (128 of 3683 cases) for Final Results of the Oslo I Study 682 SFM and 4.6% (168 of 3683 cases) for FFDM. The recall rates, the cancer detection rates, the positive predictive values, and the median tumor sizes in the Oslo I study as compared with the results of the two prior screening rounds have been presented in the first report on this study (17). A total of 31 (detection rate 0.84%) breast cancers were diagnosed in the initial screening round, including 9 (29%) ductal carcinoma in situ (DCIS) and 22 (71%) invasive carcinomas, 16 of which were ductal and 6 lobular carcinomas. Two cancers were excluded from further comparison analysis: one DCIS infiltrating the wall of a simple cyst was an incidental finding on ultrasound in a case selected by one FFDM reader, and one invasive ductal carcinoma depicted on SFM was missed at FFDM due to positioning failure. This cancer was clearly seen on repeat FFDM examination at diagnostic work-up. A total of five cancers were dismissed at consensus meetings, but the correct diagnosis was made initially because the women were called back for work-up by the other modality: two cancers were dismissed at the SFM consensus meeting and three at the FFDM meeting (Fig. 1). Ten interval cancers were diagnosed in the study population. Four of these had a true positive score in the initial reading session but were dismissed at consensus meetings. Three of the four cancers had a positive result on SFM and one cancer had a truepositive result on FFDM (Fig. 1). Histology of the three cancers with a true-positive SFM score revealed two invasive ductal carcinomas and one DCIS. The interval cancer with a true-positive initial FFDM score was an invasive lobular carcinoma. A total of 16 cancers in 15 women were diagnosed in the subsequent screening round 2 years later, 1 woman having a bilateral breast cancer. Three of these cancers had a true-positive FFDM score at initial interpretation 2 years earlier (Fig. 1), but the lesions were dismissed at FFDM consensus. Histology revealed two invasive ductal carcinomas and one invasive lobular carcinoma. Two women presenting with cancers in the subsequent screening round underwent a diagnostic work-up of the same breast 2 years earlier: one lesion proved to be a simple cyst while the second lesion proved to be a fibroadenoma. The cancer developed in another quadrant than the benign lesions in both cases; these cases are not included in the analysis. In the initial screening round, SFM depicted 28 cancers (detection rate 0.76%) and FFDM 23 cancers (detection rate 0.62%). Twenty (65%)

6 683 P. Skaane et al. Downloaded by [ ] at 14:04 19 November 2017 Fig. 1. Flow chart showing cancer detection on screen-film mammography (SFM) and full-field digital mammography (FFDM) in the inital round as well as the positive scores for each modality among the interval cancers (n510) and subsequent round cancers (n516). Of the 31 cancers diagnosed in the initial round, one FFDM detected cancer (incidental finding at work-up) and one SFM detected cancer (missed on FFDM due to positioning failure) are excluded from analysis. Two SFM detected cancers and three FFDM detected cancers with positive scores (score of 2 or higher on the 5-point rating scale for probability of cancer) were dismissed at consensus meeting but were diagnosed after recall by the other modality. In the subsequent screening round, one woman had a bilateral breast cancer. cancers were selected by both modalities. The difference in cancer detection (discordant pair eight and three in favour of SFM) was not statistically significant (McNemar P50.23). Two cancers were dismissed at SFM consensus but called back at the FFDM consensus meeting, while three cancers were dismissed at FFDM consensus but called back at SFM consensus (Fig. 1). After consensus, a total of 15 cancers were recalled on both modalities, 11 on SFM only and 5 on FFDM only (the difference in cancer detection with discordant pair 11 and 5 not statistically significant; McNemar test P50.21). There was still no difference in cancer detection rate between the two modalities after excluding the positioning failure case and the incidentally depicted cancer case from analysis. SFM detected three and FFDM one of the four interval cancers with a true-positive score in the initial reading session that were dismissed at the consensus meeting (Fig. 1). Three cancers in the subsequent screening round with true-positive scores in the initial interpretation session (but dismissed at consensus meeting) had all been selected by one of the two FFDM readers. Including all the true-positive scores for the interval cancers and the subsequent round cancers in the comparison, SFM correctly detected a total of 31 cancers as compared to 27 on FFDM, discordant pair 11 and 7 (Table 1). This overall difference in cancer detection rate is not statistically significant (McNemar test, P50.48). A true-positive score by at least one of the four independent readers occurred in 36 cancers (excluding from analysis the incidental FFDM depicted cancer and the FFDM missed cancer due to positioning failure), including 72 independent interpretations on each modality. On SFM, there were 50 of 72 (69%) true-positive individual scores as compared with 38 of 72 (53%) true-positive individual scores for FFDM. The interobserver agreement for these 36 cancers (72 interpretations) based on the 5-point rating scale showed nearly equal kappa values for SFM and FFDM (0.07 versus 0.11, Table 1. Oslo I follow-up study: comparison of cancer detection for full-field digital mammography (FFDM) and screen-film mammography (SFM), including all cancers in the initial screening round (n531), interval cancers (n510), and cancers in the subsequent screening round (n516). A positive test result (true-positive interpretation on independent double reading) means that at least one of the two readers gave a score 2 or higher on the 5-point rating scale for probability of cancer. A negative test result means that both readers scored 1 in the reading session of the initial screening round. The difference in cancer detection rate is not statistically significant (McNemar test, P-value50.48) FFDM Pos Neg Total SFM Pos Neg Total

7 Final Results of the Oslo I Study 684 Table 2. Number of individual interpretations for each radiologist on full-field digital mammography (FFDM), range and mean interpretation time (in seconds) for all FFDM interpretations, number and mean interpretation time for all FFDM cases with normal interpretations (score 1 on 5-point rating scale by both readers), and number and percentage as well as mean interpretation time for corrected normal readings (excluding outliers defined as interpretations less than 15 s or more than 180 s) Interpretation time FFDM Interpretation time for normal cases No. of FFDM All cases (s) All Corrected Reader interpretations Min. Max. Mean No. Mean No. % Mean A B C D E F G H Downloaded by [ ] at 14:04 19 November 2017 respectively). Collapsing the 5-point rating scale into a binary outcome (15negative and 2 or higher positive score), the kappa value for SFM was 0.35 (SE 0.16) and for FFDM 0.22 (SE 0.17). Comparison of SFM and FFDM interpretations by the two independent readers using the mean score for each case revealed no statistically significant difference between the two modalities (Wilcoxon signed-rank test for matched pairs; P- value50.228). The interpretation times for the radiologists at FFDM are summarized in Table 2. A wide range of reading times indicates interruptions, making several recordings unreliable. The mean reading time for the eight readers for normal examinations (category 1 by all four readers) is shown for normal cases and for corrected normal cases (excluding outliers arbitrarily defined as reading timesv15 s and w180 s). The mean interpretation time for a normal FFDM with soft-copy reading after excluding outliers varied from 31 to 63 s, with an overall mean of 45 s (Table 2). Defining outliers for a normal FFDM as reading timesv20 and w90 s showed an overall mean interpretation time of 43 s, and consequently had only a minor influence on the recorded mean reading time. The overall mean reading time for normal FFDM examinations with soft-copy reading during the Oslo I study was therefore about s. However, there was a wide range (70 97%) of normal examinations within these corrected normal reading times among the radiologists (Table 2). The interpretations of each reader for the 36 cancer cases (having a true-positive score by at least one of the four readers) and the percentage of false-negative scores on SFM and FFDM are summarized in Table 3. The number of cancer interpretations turned out to be very small for some of the radiologists. All the true-positive initial interpretations of the interval cancers and the subsequent round cancers included only category Table 3. Number of interpretations at screen-film mammography (SFM) and full-field digital mammography (FFDM), number of cancer interpretations at SFM and FFDM, number of true-positive (TP) and false-negative (FN) scores at SFM and FFDM for each radiologist. The list includes the 29 cancers from the initial screening round (one incidental finding and one positioning failure excluded) and the 7 cancers presenting as interval cancer or subsequent round cancer with a TP score by one reader in the initial screening round (a total of 36 cancers or 72 individual interpretations for each modality) Interpretations (all) All cancer interpretations Initial round cancer Subsequent cancer SFM FFDM SFM FFDM Reader SFM FFDM SFM FFDM TP FN TP FN TP FN TP FN A B C D E F G H All

8 Downloaded by [ ] at 14:04 19 November P. Skaane et al. 2 by one of the four readers, and consequently the number of false-negative scores in this group was high (Table 3). Comparisons of interpretation times for falsenegative FFDM examinations and the type of missed cancer (DCIS or invasive cancer) in the initial screening round are presented in Fig. 2. Interval cancers and subsequent round cancers are excluded, since these all had a score of only 2 by one of the four readers. The reading times for false-negative FFDM examinations were either considerably shorter or longer than for normal examinations, and all reading times for missed DCIS were remarkably short compared with normal examinations (Fig. 2). Comparison of the true-positive and false-negative SFM and FFDM interpretations and the mammographic features for the 36 cancers is presented in Table 4. Overall, there were 31% (22 of 72) falsenegative interpretations at SFM as compared to 47% (34 of 72) at FFDM. There is a high proportion of false-negative FFDM interpretations for spiculated masses of 63% (10 of 16 cases) and of 44% (8 of 18 DCIS) for microcalcifications (Table 4). Discussion One main purpose of this study was follow-up of the study population for more than 2 years so that interval cancers and subsequent screening round cancers with initial true-positive scores could be included in the comparison of SFM and FFDM. Cancer detection based on positive scores and diagnosis at initial diagnostic work-up has previously been reported showing no significant difference between the two modalities (17). Four interval cancers and three subsequent round cancers initially had a true-positive score but had been dismissed at consensus: four true-positive FFDM scores and three true-positive SFM scores (Fig. 1). Including these seven cancers in the comparison analysis, there is still no statistically significant difference in the overall cancer detection rate between SFM and FFDM (Table 1). This non-significant difference in cancer detection between the two modalities applies regardless of whether comparison is based on positive scores in the interpretation session, on diagnosed cancers at work-up after consensus, or by including Fig. 2. Scatter plot showing the reading times (in seconds) of the eight radiologists (A H) for their false-negative (FN) FFDM soft-copy interpretations in the initial screening round. The horizontal dotted line is the overall mean interpretation time for normal examinations (score 1 on the 5-point rating scale by both readers) after excluding the outliers. The short lines show the mean interpretation time for normal examinations for each reader. Black triangles represent the FN results for ductal carcinoma in situ DCIS (microcalcifications) and round black marks the FN readings for invasive cancers.

9 Final Results of the Oslo I Study 686 Table 4. Comparison of mammographic features and the true-positive (TP) and false-negative (FN) interpretations for full-field digital mammography (FFDM) and screen-film mammography (SFM). Only cancers with a TP score by at least one of the four independent readers (n536) are included. Two cancers from the initial screening round (one incidental finding depicted at FFDM and one cancer overlooked at FFDM due to positioning failure) are excluded from the analysis. The number of interpretations and not the number of cancer cases are listed (i.e. two interpretations for each modality for each cancer case due to independent double reading for SFM as well as for FFDM Initial round cancer Subsequent cancer SFM FFDM SFM FFDM Mammographic features No. of interpretations TP FN TP FN TP FN TP FN Ill-defined mass Spiculated mass Distortion/asymmetric density Calcifications only Density + calcifications All Downloaded by [ ] at 14:04 19 November 2017 subsequent cancers with initial true-positive scores in the analysis. A total of 12 cancers, including the interval and subsequent round cancers with true-positive score at the initial reading session, were dismissed at consensus meetings. Five were dismissed at SFM and seven at FFDM consensus. Independent double reading, as used in our screening program, can increase the cancer detection rate but may have a double impact on call-back rates depending on the recall policy used (4, 9). We have a consensus (or arbitration) meeting before any final decision is taken as to whether a woman selected by one or both readers should be recalled for diagnostic workup or dismissed and sent back to the screening program. Unilateral recall, i.e. recall if either reader has a positive score, is not used. Although double reading by consensus or arbitration in general achieves an increase in cancer detection along with a reduction of women recalled for diagnostic work-up (4), cancers may be dismissed using this practice. All lesions subsequently proved to be malignant may not be detected with panel consensus or arbitration (5). Four circumstances might have had a bearing on the remarkably high number of 12 cancers dismissed at our consensus: first, we never practice shortterm follow-up for probably benign lesions. The woman is either called back immediately for diagnostic work-up or she goes back to the screening program with a new examination scheduled 2 years later. Second, according to the guidelines of the NBCSP, the recall rate should be below 5%. Efforts to lower the false-positive recall rates may result in missed cancers. Recall rates between 4.9% and 5.5% have been reported achieving the best trade-off of sensitivity and positive predictive value (23). Third, there is a learning curve effect, at least regarding the FFDM dismissed cases, since the radiologists were not trained with the same level of familiarity with soft-copy reading. Fourth, interobserver variation is one of the greatest challenges in mammography screening (1, 6). Inconsistencies between individual observers are greatest for interpretation of probably benign and suspicious abnormalities, and for assessments including the intervention threshold (2, 7, 19). Failure to act on non-specific minimal sign lesions, which constitute about 38% of interval and screen-detected cancers (21), does not necessarily constitute interpretation below a reasonable standard (10). A second main purpose of this study was to analyze the false-negative digital interpretations, since more cancers had been overlooked at FFDM with soft-copy reading. We do not believe there is any flaw in the study design that might have disfavored one of the two modalities investigated. Previous mammograms for comparison were not offered in the reading sessions in order to avoid interpretation bias, since previous SFM but no prior FFDM would have been available. Previous mammograms do not significantly increase sensitivity (cancer detection), although specificity is improved (3, 20). In any case, the consequences of no prior mammograms for comparison should be equal for both modalities. However, we focus on two possible weaknesses of our study which might have disfavored FFDM (17): insufficient experience of the radiologists in soft-copy reading and a suboptimal reading environment. All radiologists were experienced in SFM screening but they were not trained to the same level of familiarity with digital soft-copy reading before the project started. The FFDM interpretations were carried out in a room with other activities, and interruptions were obviously a problem, as reflected by the extremely wide range of recorded reading times (Table 2). On the other hand, in the Oslo II study the radiologists had experience in soft-copy reading, and the FFDM interpretations were carried out in dedicated,

10 Downloaded by [ ] at 14:04 19 November P. Skaane et al. darkened, and quiet environments. The higher cancer detection rate for FFDM as compared to SFM in the Oslo II study, close to statistical significance for the group aged between 50 and 69, clearly indicates the importance of training (learning curve effect) and dedicated reading environments for soft-copy display (16). A previous study showed no dominant cause accounting for overlooked cancers on one modality and detected on the other (11). This is in accordance with our side-by-side feature analysis for cancer conspicuity of SFM and FFDM showing no difference between the two modalities (17). The appearances of some cancers were different with the two techniques probably because of slight variation in projection during imaging. This study showed neither a difference in cancer detection nor a significant difference in the mean levels of the double reading scores for SFM and FFDM. Nevertheless, it is noteworthy that more cancers were missed at FFDM compared with SFM (Table 1), and the difference was even more impressive when comparing individual interpretations and not the summary of independent double reading. A total of 69% (50 of 72) of interpretations were true positive at SFM as compared with 53% (38 of 72) true-positive score at FFDM (Table 3). Six cancers were missed at FFDM by both readers but depicted at SFM by both readers, as compared with one cancer depicted by both readers at FFDM but overlooked by both readers at SFM. Since positioning variability may account for overlooked cancer on one modality, a missed cancer by one reader could be defined as a tumor correctly detected by three of the four independent readers. Using such a definition, one cancer was missed by one reader on SFM but detected by the second SFM reader and both FFDM readers, as compared with six cancers missed by one FFDM reader but correctly detected by the second FFDM reader and both radiologists on SFM interpretation. Mammographic features of the cancers with falsenegative SFM and FFDM scores are summarized in Table 4. It is noteworthy that several cancers presenting as spiculated mass or microcalcifications only (DCIS) were overlooked on FFDM. A total of 44% of the DCIS interpretations and 63% of spiculated mass cancers had false-negative ratings on FFDM with soft-copy reading. For the other cancers, there were only minor differences between the false-negative interpretations at the two modalities (Table 4). The high number of overlooked DCIS at FFDM is noteworthy, since experimental clinical studies have shown that FFDM provides better image quality than SFM in patients with microcalcifications and has a higher sensitivity and a higher reliability in the characterization of microcalcifications (8). Analysis of our false-negative FFDM interpretations emphasizes other important aspects of softcopy reading: display protocol and systematic reviewing of the images. No difference was found in the accuracy of cancer classification between hard copy and soft-copy reading of digital mammography in a previous study (13). The reading protocol used for image display would most likely be crucial to the success of FFDM with soft-copy reading. The rationale for our display protocol was that if postprocessing (window-level adjustments, zooming, inversion) is carried out at only one of the three steps, for whatever reason, this is best done on MLO images, since most of the breast parenchyma is seen on these views. The mean FFDM interpretation time of 45 s for normal examinations in our study is comparable to the range of to more than 100 s reported for soft-copy display in experimental studies (14, 15, 18, 22), although caution is needed when comparing results from experimental studies with reading times from our daily practice in a screening setting with a high work-flow. Nevertheless, there are two notable results concerning the recorded soft-copy interpretation times: inconsistencies in reading times for normal examinations (Table 2) and divergence of the recorded reading times for false-negative FFDM interpretations from that of normal examinations (Fig. 2). Normal examinations within an arbitrarily defined range (excluding the outliers) of normal interpretation times varied among readers from 70% to 97% (Table 2). Since more than 95% of cases in the NBCSP are expected to be considered normal by both readers, all examinations are automatically offered as normal on the screen in order to make the reading session as convenient as possible to the reader. The reader has only to confirm the normal result using the light pen or one click with the mouse. Given the high number of about examinations in our daily batch reading session, one would expect the percentage of normal cases within a suggested normal reading time to be consistently high for all readers, although the mean reading time would vary from fast to slow readers. However, only three radiologists interpreted more than 90% of normal examinations within the suggested normal reading time, and the high number of outliers may indicate a lower consistency regarding the systematic use of the image display protocol (Table 2).

11 Downloaded by [ ] at 14:04 19 November 2017 The reading times for false-negative FFDM interpretations compared with normal reading times for the eight radiologists are presented in Fig. 2. An important question is whether unsystematic use of the image display protocol, as expressed by the high number of outliers, may be of importance for the false-negative FFDM interpretations. The scatter plot shows that most of the false-negative FFDM cases either have considerably longer or considerably shorter reading times compared with normal examinations (Fig. 2). A case with a recorded reading time of 3 s is obviously not representative and might be explained by finished interpretation prior to fetching the woman on the RIS screen with the light pen (indirectly recorded reading times). In retrospect, it is not possible to give a definite reason why a cancer was missed at the interpretation session. Nevertheless, our results may indicate that at least some of the cancers with recorded interpretation times considerably longer than for normal examinations may have been seen by the readers but dismissed after post-processing ( decision errors ), whereas some cancers with considerably shorter reading times represent perception errors as a consequence of (too) fast interpretation. It is noticeable that the opposite breast was selected in 21% (7 of 34) of FFDM missed cancer cases. We have a side-based recall (i.e. the reader has to select the right or left breast if an abnormality is present), and the false side could have been selected due to the complexity of the soft-copy reading environment. It is noteworthy that for most of the overlooked DCIS (presenting as microcalcifications only) interpretation times shorter than mean reading time for normal examinations were recorded (Fig. 2). Postprocessing including electronic zooming (and occasionally window-level adjustments) is occasionally mandatory in order to detect fine granular microcalcifications, especially if the DCIS is located in an area with dense breast parenchyma. Proper postprocessing would not have been possible within the short interpretation time recorded for some of the DCIS overlooked on soft-copy reading in our study (Fig. 2). In conclusion, our results from the Oslo I followup study confirm the previous report that there is no significant difference in the cancer detection rate between SFM and FFDM with soft-copy reading. Analysis of the false-negative FFDM interpretations indicates, taking into consideration the results of the Oslo II study, that close attention must be paid to the design of efficient work stations, proper reader training, optimal reading environments, and systematic use of proper image display protocols in order to succeed with FFDM using soft-copy reading. References Final Results of the Oslo I Study Beam CA, Layde PM, Sullivan DC. Variability in the interpretation of screening mammograms by US radiologists. Arch Intern Med 1996;156: Berg WA, Campassi C, Langenberg P, Sexton MJ. Breast imaging reporting and data system: inter- and intraobserver variability in feature analysis and final assessment. Am J Roentgenol 2000;174: Burnside ES, Sickles EA, Sohlich RE, Dee KE. Differential value of comparison with previous examinations in diagnostic versus screening mammography. Am J Roentgenol 2002;179: Dinnes J, Moss S, Melia J, Blanks R, Song F, Kleijnen J. Effectiveness and cost-effectiveness of double reading of mammograms in breast cancer screening: findings of a systematic review. The Breast 2001;10: Duijm LEM, Groenewoud JH, Hendriks JHCL, de Koning HJ. Independent double reading of screening mammograms in the Netherlands: effect of arbitration following reader disagreements. Radiology 2004;231: Elmore JG, Wells CK, Lee CH, Howard DH, Feinstein AR. Variability in radiologists interpretations of mammograms. N Engl J Med 1994;331: Elmore JG, Nakano CY, Koepsell TD, Desnick LM, D Orsi CJ, Ransohoff DF. International variation in screening mammography interpretations in communitybased programs. J Natl Cancer Inst 2003;95: Fischer U, Baum F, Obenauer S, Luftner-Nagel S, von Heyden D, Vosshenrich R, et al. Comparative study in patients with microcalcifications: full-field digital mammography vs screen-film mammography. Eur Radiol 2002;12: Harvey SC, Geller B, Oppenheimer RG, Pinet M, Riddell L, Garra B. Increase in cancer detection and recall rates with independent double interpretation of screening mammography. Am J Roentgenol 2003;180: Ikeda DM, Birdwell RL, O Shaughnessy KF, Brenner RJ, Sickles EA. Analysis of 172 subtle findings on prior normal mammograms in women with breast cancer detected at follow-up screening. Radiology 2003;226: Lewin JM, D Orsi CJ, Hendrick RE, Moss LJ, Isaacs PK, Karellas A, et al. Clinical comparison of full-field digital mammography and screen-film mammography for detection of breast cancer. Am J Roentgenol 2002;179: Lewin JM, Hendrick RE, D Orsi CJ, Isaacs PK, Moss LJ, Karellas A, et al. Comparison of full-field digital mammography with screen-film mammography for cancer detection: results of 4,945 paired examinations. Radiology 2001;218: Obenauer S, Hermann KP, Marten K, Luftner-Nagel S, von Heyden D, Skaane P, et al. Soft copy versus hard copy reading in digital mammography. J Digit Imaging 2003;16: O Riordan E, Bukhanov K, Muradali D, Goldberg F. A comparison of reporting times for analog vs digital mammography. Radiology 2000;217(P):200.

12 Downloaded by [ ] at 14:04 19 November P. Skaane et al. 15. Pisano ED, Cole EB, Kistner EO, Muller KE, Hemminger BM, Brown ML, et al. Interpretation of digital mammograms: comparison of speed and accuracy of soft-copy versus printed-film display. Radiology 2002;223: Skaane P, Skjennald A. Screen-film mammography versus full-field digital mammography with soft-copy reading: randomized trial in a population-based screening program The Oslo II study. Radiology 2004;232: Skaane P, Young K, Skjennald A. Population-based mammography screening: comparison of screen-film and full-field digital mammography with soft-copy reading Oslo I study. Radiology 2003;229: Solari M, Berns EA, Hendrick RE, Wolfman JA, Willis W, Segal L, et al. Comparison of interpretation times for screening exams between soft copy full-field digital mammography and hard copy screen-film mammography. Am J Roentgenol 2004;182 Suppl: Taplin SH, Ichikawa LE, Kerlikowske K, Ernster VL, Rosenberg RD, Yankaskas BC, et al. Concordance of breast imaging reporting and data system assessments and management recommendations in screening mammography. Radiology 2002;222: Thurfjell MG, Vitak B, Azavedo E, Svane G, Thurfjell E. Effect on sensitivity and specificity of mammography screening with or without comparison of old mammograms. Acta Radiol 2000;41: van Dijck JAAM, Verbeek ALM, Hendriks JHCL, Holland R. The current detectability of breast cancer in a mammographic screening program. A review of the previous mammograms of interval and screen-detected cancers. Cancer 1993;72: Wedekind N, Roelofs T, van Woudenberg S, Beck C, Del Turco MR, Evertsz CJ. Impact of training on softcopy reading of full field digital mammograms: a study from the European Screen-trial project. Radiology 2003;229(P): Yankaskas BC, Cleveland RJ, Schell MJ, Kozar R. Association of recall rates with sensitivity and positive predictive values of screening mammography. Am J Roentgenol 2001;177:543 9.