Differential diagnosis of adnexal masses: sequential use of the risk of malignancy index and HistoScanning, a novel computer-aided diagnostic tool

Transcription

1 Ultrasound Obstet Gynecol 2012; 39: Published online in Wiley Online Library (wileyonlinelibrary.com). DOI: /uog.9079 Differential diagnosis of adnexal masses: sequential use of the risk of malignancy index and HistoScanning, a novel computer-aided diagnostic tool E. VAES*, R. MANCHANDA, P. AUTIER, R. NIR, D. NIR, H. BLEIBERG**, A. ROBERT* and U. MENON *Université Catholique de Louvain, Institut de Recherche Experimentale et Clinique, Epidemiology and Biostatistics Research Center, Clos Chapelle-aux-champs, Brussels, Belgium; Max Planck Institute of Biophysics, Department of Molecular Neurogenetics, Frankfurt, Germany; Gynaecological Oncology, UCL EGA Institute for Women s Health, London, UK; International Prevention Research Institute IPRI, Ecully (Lyon), France; Advanced Medical Diagnostics, Waterloo, Belgium; **Jules Bordet Institute, Brussels, Belgium KEYWORDS: differential diagnosis; HistoScanning; ovarian cancer; RMI; sequential strategy; ultrasound ABSTRACT INTRODUCTION Objective To assess the value of ovarian Histo- Scanning TM, a novel computerized technique for interpreting ultrasound data, in combination with the risk of malignancy index (RMI) in improving triage for women with adnexal masses. Methods RMI indices were assessed in 199 women enrolled in a prospective study to investigate the use of HistoScanning. Ultrasound scores were obtained by blinded analysis of archived images. The following sequential test was developed: HistoScanning was modeled as a second-line test for RMI between a lower cut-off and an upper cut-off. The optimal combination of these cut-offs that together maximized the Youden index (Sensitivity + Specificity 1) was determined. Results Using RMI at the standard cut-off value of 250 resulted in a sensitivity of 74% and a specificity of 86%. When RMI was combined with HistoScanning, the highest accuracy was achieved by using HistoScanning as a sequential second-line test for patients with RMI values between 105 and At these cut-off values, sequential use of RMI and HistoScanning resulted in mean sensitivity and specificity estimates of 88% and 95%, respectively. Conclusions Our data suggest that HistoScanning may have the potential to improve the diagnostic accuracy of RMI, which could result in better triage for women with adnexal masses. Further prospective validation is warranted. Copyright 2011 ISUOG. Published by John Wiley & Sons, Ltd. Ovarian cancer is the leading cause of death from gynecological malignancies. This is often attributed to its silent progression that leads to diagnosis in advanced stages (International Federation of Gynecology and Obstetrics (FIGO) Stage III/IV), which have 5-year survival rates of only 6 22% 1. There is increasing evidence that outcomes can be improved if cancer patients are managed by trained gynecological oncologists in multidisciplinary teams 2 4. This depends on the accurate differential diagnosis of adnexal masses and timely referral of patients with malignant masses to cancer centers, while general gynecologists at district hospitals deal with benign lesions. Preoperative diagnosis of malignant ovarian neoplasms usually involves pelvic examination followed by transvaginal ultrasonography (TVS) and assessment of the serum tumor marker CA 125. Both serum CA 125 and TVS, when used alone, have high sensitivity but limited specificity for the differential diagnosis of adnexal masses 5. Morphology-based indices, Doppler blood-flow indices and power Doppler have been investigated to see if they can improve performance characteristics. Addition of CA 125 did not improve the performance of ultrasound morphology-based logistic regression models 6,7. Subjective evaluation (pattern recognition) by an experienced examiner has been found to improve specificity and to be better than either CA 125 alone 8,9 or mathematical logistic regression models 10 in discriminating between benign and malignant adnexal masses. Recently, the International Ovarian Tumor Analysis Group reported 10 simple rules (based on morphological description) for initial triage of Correspondence to: Dr E. Vaes, Université Catholique de Louvain, IREC - EPID B , Clos Chapelle-aux-champs, 30, BE-1200 Brussels, Belgium ( evelien.vaes@uclouvain.be) Accepted: 13 June 2011 Copyright 2011 ISUOG. Published by John Wiley & Sons, Ltd. ORIGINAL PAPER

2 92 Vaes et al. adnexal masses. These simple rules could correctly diagnose three out of four ovarian masses with a sensitivity of 92% (95% CI, 89 94%) and specificity of 96% (95% CI, 94 97%). Subjective assessment by an experienced ultrasonographer was found to improve mathematical models for diagnosing the remaining 25% tumors for which these simple rules were inconclusive 11. The risk of malignancy index (RMI) 12 is a standardized index commonly used for clinical evaluation of patients with an adnexal mass. In the UK it is routinely used to triage women with ovarian masses along appropriate referral pathways The RMI compares well with most logistic regression models or artificial neural networks 17, and has been advocated as the prediction model of choice in preoperative assessment of adnexal masses 18.TheRMI has been validated in many European countries 13,15,19,20. Ovarian HistoScanning TM (Advanced Medical Diagnostics, Waterloo, Belgium) is a new ultrasound-based computerized tissue-characterization technique 21.Inits present form, ovarian HistoScanning analyzes ultrasound gray-scale voxel data. This analysis is based on the fact that cancerous processes induce structural changes in the texture of the tissue, which results in physical changes in the reflected ultrasound waves. Different tissue characterization algorithms have been optimized to quantify such changes in the backscattered waves 22. Selection of the appropriate algorithms for a specific organ and their training is an empirical process. For ovarian Histo- Scanning, three different characterization algorithms were selected and trained during clinical development studies in (unpubl. data). In this paper, we explore the possibility of using ovarian HistoScanning as a sequential test in women with RMI values not associated with either very low or very high probability of cancer (Figure 1) in order to improve the clinical triage of patients with adnexal masses. RMI < LC: no referral to specialized center Adnexal mass RMI assessment (M (CA 125) U) LC RMI < UC: HistoScanning assessment RMI UC: referral to specialized center PATIENTS AND METHODS Patients Data for this analysis were obtained from a cohort of patients, who were originally recruited to a prospective multicenter study evaluating the accuracy of ovarian HistoScanning 21 ; a detailed description of the cohort and recruitment process has been given elsewhere 21. Ovarian HistoScanning results were not used for patient management. Basic information collected included age, menopausal status, detailed histopathology of surgical specimens and stage of the cancers. At the time of prospective data collection, clinicians were asked to classify the ovary as positive (malignant) or negative (benign) based on TVS findings. RMI was not calculated at the time. Details of ovarian morphology at the time of initial TVS were not retained on the database, however, images were archived and stored. This makes it possible to retrospectively calculate RMI values for patients for whom CA 125 data were available. Patients from the cohort for whom serum CA 125 levels were available (n = 199) were selected for this analysis. Ultrasonography A brief description of the ultrasound details that are important for this work are given below, but more details are described in the paper by Lucidarme et al. 21.Atthe time of the original study eligible women underwent TVS shortly before surgery by an expert sonographer with more than 10 years scanning experience. The Voluson 730 Expert scanner equipped with a RIC5-9, 5 9- MHz Real Time 4D broadband endocavitary transducer (GE Medical Systems, Zipf, Austria) was used at all participating centers. The examination involved conventional two-dimensional (2D)-TVS for identifying ovaries and/or adnexal masses followed by a standardized three-dimensional (3D)-TVS procedure to capture the 3D image of all structures. If necessary, several 3D acquisitions were made to cover a mass. Ovaries and lesions were annotated and marked as regions of interest on the 2D scan by the individual operators. All acquired data were anonymized, saved on a CD-ROM and sent to the study coordinator. An independent expert matched the 2D annotated images to the structure depicted by the 3D-TVS data to determine theextentofthevolume(s)ofinterest(voi)tobeanalyzed by HistoScanning. Ovarian HistoScanning HistoScanning < HSCU: no referral to specialized center HistoScanning HSCU: referral to specialized center Figure 1 Flow-chart showing strategy for differential diagnosis of adnexal masses. LC, lower cut-off; HSCU, HistoScanning cut-off; M, menopausal status; RMI, risk of malignancy index; U, ultrasound score; UC, upper cut-off. See text for details of M and U. HistoScanning analysis was applied to the raw voxel data (gray-scale values) that had been saved on the CD- ROMs. HistoScanning successively analyzed the VOIs that were indicated on the 3D scan by the operator. The characterization algorithms calculated a score value for each VOI in a scan. This was a continuous value ranging from 0 to 125. In this work, the final HistoScanning

3 Differentiating adnexal masses by RMI plus HistoScanning 93 result for one individual patient was the maximum of this score over all available VOIs for that patient. This analysis was blinded with regard to the ultrasonographers interpretation, CA 125 value and pathology results. RMI calculations All archived 2D- and 3D-ultrasound images were reviewed at University College, London by two independent experienced ultrasonographers, who were blinded to the HistoScanning, CA 125 and pathology results and assessed each ultrasound image for the presence of a number of key morphological features (multi-locularity of any cyst, presence of solid areas, evidence of metastases, presence of ascites and bilaterality of lesions) necessary for the RMI 12. The RMI is defined as the product of menopausal status (M), CA 125 level and ultrasound score (U). The ultrasound score is based on the five features described above, each of which scores one point. At present there are three versions of the RMI, each varying slightly in the method of scoring. The original RMI (RMI 1 ) was described by Jacobs et al. 12 and two modifications (RMI 2 and RMI 3 ) have subsequently been described by Tingulstad et al. 15,19.InRMI 1,U= 0 for an ultrasound score of 0, U = 1 for an ultrasound score of 1 and U = 3 for an ultrasound score 2; M = 1 for premenopausal status and M = 3 for postmenopausal status. In RMI 2, M = 4 for postmenopausal status; U = 1 for an ultrasound score of 1andU= 4 for an ultrasound-score 2. The changes in RMI 3 (compared with RMI 2 ) include M = 3 instead of 4 for postmenopausal status and U = 3 (instead of 4) when ultrasound score 2. Analysis Our aim was to develop a sequential strategy using the RMI and ovarian HistoScanning that would enhance the accuracy of the RMI for differential diagnosis. The following scheme was adopted for this purpose: patients with an RMI below a lower cut-off (LC) were considered to have a negative (non-malignant) test outcome and patients with an RMI an upper cut-off (UC) were considered to have a positive (malignant) test outcome (Figure 1). The HistoScanning score was used to determine the test outcome for patients with an RMI value between the upper and lower cut offs (LC RMI < UC). The outcome was positive when the HistoScanning score was greater than a cut-off point (the HistoScanning cut-off (HSCU)) and negative otherwise. Subsequently, we identified the optimal combination of the three cut-offs, LC, UC and HSCU, that together would give the highest sensitivity and specificity for this sequential test (RMI followed by HistoScanning). This combination of cut-offs was then validated on an independent set of data not used for training. A training set of 70% of the patients, selected randomly, was used to identify the three cut-offs LC, UC and HSCU, which was done by varying LC and UC for RMI and HSCU simultaneously in predefined ranges that seemed appropriate. The LC was varied between 5 and 265 with steps of 20, the UC between 500 and 2500 with steps of 50, and the HSCU was considered in its full range with steps of 5. Then the best combination was defined as the one that maximized the Youden Index (sensitivity + specificity 1). This operating point was validated by calculating sensitivity and specificity in the remaining 30% of patients (testing set). To test the variability that might exist in our data set, this random subsampling process was repeated 100 times so that 100 randomly selected training and testing sets were created 23. At every iteration step the testing set did not contribute to the optimization of the operating point and served as an independent assessment. Statistical analysis was performed using SAS version 9.1 (SAS Institute Inc., Cary, NC, USA) and Matlab version R2008a (Mathworks, Natick, MA, USA). Means or medians of continuous variables were compared using ANOVA with F-test or the Kruskal Wallis test and the chi-square test was used to compare proportions. To measure the diagnostic accuracy, sensitivities and specificities were determined at fixed operating points. Receiver operating characteristics (ROC) curves were constructed and the area under the ROC curve (AUC) was calculated using the trapezoidal rule. AUCs were compared using the nonparametric approach of DeLong et al. 24. RESULTS Patient characteristics Of the 199 patients, two were excluded owing to lack of agreement between the pathology report and the study case report form. In the remaining 197 patients there was no significant age difference between women found to have normal ovaries and those with malignant or benign growths. However, there were more postmenopausal women in the normal and benign groups than in the malignant group. Serum CA 125 levels were significantly higher in women with malignant masses (Table 1). The 197 patients had 291 adnexal masses, of which 125 were non-malignant and 166 were malignant. Details of ovarian histology are given in Table 2. The majority (69%) of malignant tumors were primary ovarian cancers and borderline ovarian tumors. 37% of all ovarian cancers and 94% of borderline ovarian tumors were early-stage cancers (FIGO stages I and II). Of the total of 166 malignant tumors, 69% were of epithelial subtype. RMI At University College, London, two sets of ultrasound scores for each of the 197 patients were obtained independently from the two sonographers. The two RMI 1 -scores that resulted from multiplying these scores with CA 125 values and menopausal status showed very similar ROC curves and AUCs (Figure 2) in our patient population. Therefore it was decided to use the RMI

4 94 Vaes et al. Table 1 Patient characteristics (n = 197) Variable Malignant (n = 98) Benign (n = 72) Normal (n = 27) P Age (years, mean ± SD) 57 ± ± ± CA 125 level (U/mL, median (range)) 2323 ( ) 12 (4 487) 17 (3 356) < Postmenopausal (%) Table 2 Histopathology of tumors in the study (n = 291) 1.0 Type of tumor n (%) Non-malignant (n = 125) Ovarian cyst 30 (24.0) Cystadenoma 28 (22.4) Cystadenofibroma 15 (12.0) Corpus albicans 14 (11.2) Endometriosis 8 (6.4) Corpus luteum cyst 7 (5.6) Fibroma 6 (4.8) Brenner tumor 6 (4.8) Teratoma 4 (3.2) Thecoma 2 (1.6) Hemorrhagic adnexal cyst 1 (0.8) Unknown 4 (3.2) Malignant (n = 166) Primary ovarian tumor 89 (53.6) Borderline ovarian tumor 25 (15.1) Secondary (metastatic) tumor 16 (9.6) Peritoneal tumor 15 (9.0) Cancer of an organ invading the pelvis 4 (2.4) Unknown 17 (10.2) based on the ultrasound score of the second sonographer (random choice) for further analysis. The ROC curves for RMI 1, RMI 2 and RMI 3 were obtained and the respective AUCs were estimated as 0.85 (95% CI, ), 0.87 (95% CI, ) and 0.88 (95% CI, ), respectively. The difference between the three AUCs was not statistically significant (P = 0.99). Additionally, the ROCs of the three RMIs hardly changed when the 27 normal patients were not included in the sample, thus confirming that the strength of the RMI is its ability to differentiate between malignant and non-malignant adnexal masses. This can also be seen from the empirical histograms of the log-rmi 2 score (Figure 3), in which the normal and benign groups completely overlap. Sensitivity and specificity of the three RMIs for the 197 patients were evaluated. RMI 2 had the best sensitivity, at a cut-off of 250, of 82% (95% CI, 73 89%), with a specificity of 81% (95% CI, 72 88%). RMI 1 and RMI 3 at a cut-off of 250 had sensitivities of 68% (95% CI, 58 77%) and 74% (95% CI, 64 82%), respectively, and specificities of 89% (95% CI, 81 94%) and 86% (95% CI, 77 92%), respectively. Sensitivity Specificity Figure 2 Receiver operating characteristics (ROC) curves of risk of malignancy index 1 (RMI 1 ) for two independent sonographers., Sonographer 1; area under ROC curve (AUC), 0.85;, Sonographer 2; AUC, Frequency Log (RMI 2 ) Figure 3 Empirical histograms of the log-risk of malignancy index 2 (log (RMI 2 )) score for normal ( ), benign ( ) and malignant ( ) pathology groups. Sequential use of RMI and HistoScanning Table 3 presents the results of the 100 resampling processes carried out to find the combination of the three cut-offs (LC, UC and HSCU) that maximized the Youden Index. All combinations of RMI with HistoScanning showed improvements over RMI alone.

5 Differentiating adnexal masses by RMI plus HistoScanning 95 Table 3 Results of a repeated random subsampling procedure on RMI 1,RMI 2 and RMI 3 supplemented with HistoScanning to find the operating point that maximizes the Youden Index Training set Testing set Test Cut-offs (LC, UC, HSCU)* OPF (%) Sensitivity (%, mean (95% DI)) Specificity (%, mean (95% DI)) RMI 1 followed by HistoScanning (25, 2100, 25) (78 97) 84 (73 94) RMI 2 followed by HistoScanning (165, 2050, 20) (76 91) 93 (86 99) RMI 3 followed by HistoScanning (105, 2100, 20) (79 98) 95 (87 99) *Combination of the three cut-off points. Mean sensitivity/specificity with 95% distribution intervals (DI) over the number of resamplings in which the operating point maximized the Youden index on the training data (= OPF). This served as a validation of the retained operating point. OPF, operating point frequency, i.e. percentage of resamplings in which the three cut-off points maximized the Youden index. Adnexal mass, n = 59 Normal, n = 6 Benign, n = 23 Borderline cancer, n = 8 Invasive cancer, n = 22 RMI 3 assessment (M (CA 125) U) RMI 3 < 105: No referral to specialized center 105 RMI 3 < 2100: HistoScanning assessment RMI : Referral to specialized center Normal, n = 3 Normal, n = 3 Normal, n = 0 Benign, n = 15 Benign, n = 8 Benign, n = 0 Borderline cancer, n = 3 Borderline cancer, n = 5 Borderline cancer, n = 0 Invasive cancer, n = 1 Invasive cancer, n = 14 Invasive cancer, n = 7 HistoScanning < 20: No referral to specialized center Normal, n = 3 Benign, n = 5 Borderline cancer, n = 0 Invasive cancer, n = 0 HistoScanning 20: Referral to specialized center Normal, n = 0 Benign, n = 3 Borderline cancer, n = 5 Invasive cancer, n = 14 Figure 4 Strategy using the risk of malignancy index 3 (RMI 3 ) and ovarian HistoScanning sequentially for differential diagnosis of adnexal masses with results from one testing set of 59 patients, showing how they were divided along the proposed pathway. M, menopausal status; U, ultrasound score. The strategy combining RMI 3 and HistoScanning led to the highest mean accuracy in the iteratively selected testing sets (Table 3). For RMI 3 with HistoScanning, the best combination of the three cut-offs (LC, UC and HSCU) was 105, 2100 and 20, with mean sensitivity and specificity estimates of 88% and 95%, respectively (testing). Working example illustration Since the sequential use of RMI 3 with HistoScanning achieved the best performance at optimized cut-offs of 105, 2100 and 20, we applied this sequential strategy on 59 patients from one independent testing set (one out of the 100 splits) (Figure 4). Application of the RMI 3 would directly find 22 women with low probability (RMI 3 < 105) and seven women with high probability (RMI ) for adnexal cancer. The invasive carcinoma found among patients with RMI 3 < 105 was in a 44-year-old woman with CA 125 of 36 U/mL and RMI 3 of 36, diagnosed with ovarian metastases from breast cancer in both ovaries. For the 30 women in the RMI 3 gray zone (105 RMI 3 < 2100), HistoScanning could further identify 22 women who should be referred to a specialized center (women with HistoScanning score 20). These 22 women included 19 with cancer and three with benign tumors. The eight other women in the RMI 3 gray zone had HistoScanning values < 20 and could be followed up by their own doctor. Of these eight women, three were found to have normal ovaries and five had benign tumors at histology.

6 96 Vaes et al. DISCUSSION This is the first study to explore the use of the novel ultrasound-based technique ovarian HistoScanning to supplement the RMI for the differential diagnosis of adnexal masses. The study was able to identify an RMI gray-zone in which HistoScanning could be used as a second-line test to improve performance characteristics. The highest accuracy (mean sensitivity and specificity of 88% and 95%, respectively) was achieved using a sequential strategy of RMI 3 with HistoScanning as a second-line test for RMI 3 cut-off values of 105 and < In this study there was no significant difference in the performance characteristics of RMI 1,RMI 2 and RMI 3, which is consistent with earlier reports 25,26. Only one study has reported improved accuracy of RMI 2 over RMI External validation of the sequential test requires an independent testing set but this was impossible at the time. A possible option would have been to do one single split of the data in the training and testing sets (internal validation). However, given the high proportion of malignant cases and the moderate sample size, a single randomly chosen testing set would have been subject to chance (accuracy results in favor or not in favor of the new method). In order to minimize this selection bias, we used an iterative approach that permitted understanding of the variability that might arise from data representative of the population under study. The cut-off determined using this method minimizes selection bias, is more robust and is likely to represent a true finding and be reproducible. However, the drawback is that the results presented on one single testing set based on one single random data split (as in Figure 4) should be treated with caution. Although we are cautiously optimistic regarding the findings using internal validation, further prospective validation on a totally independent data set is awaited. While the data used in this work were prospectively collected, they were obtained from a clinical study designed to investigate the accuracy of HistoScanning alone, without regard to RMI, which was analyzed retrospectively for our study. Two experienced sonographers reviewed archived 2D- and 3D-TVS images to derive morphological scores. This could be considered suboptimal. However, the images were all acquired by experts with at least 10 years experience, and assessing such archived ultrasound images has been noted to improve observer accuracy 27. This is confirmed by the finding that the range of RMI scores for these patients is in line with other RMI validation studies 15,19,28. The quality of subjective assessment of adnexal masses on ultrasound images varies widely, depending on operator expertise 27,29. This variability may come from two sources definition of the region of interest and interpretation of the morphological characteristics of the selected region. Ovarian HistoScanning, being an automated computerized technique, has the potential to significantly reduce observer variability in image interpretation. Even so, ovarian HistoScanning software is not completely without user dependency, as it requires identification of the VOI, which falls under the first source of variability. Unfortunately, within the context of the design of the clinical trial, it was impossible to assess interuser variability related to the definition of the VOI. For a future application of our proposed strategy, assessment of masses on the ultrasound image as well as initiation of the HistoScanning analysis on this image should ideally be done by the same operator during the same examination. Addressing the user/expertise dependency in such a situation would thus be important. Lucidarme et al. 21 showed that the sensitivity of TVS increases from 66 to 92% when TVS and HistoScanning are combined in parallel. However, RMI remains the current standard for differential diagnosis in many centers. Our study is thus novel as, for the first time, it shows that HistoScanning can be used as a sequential secondline test to improve performance of RMI and more accurately triage women with adnexal masses. Van Trappen et al. 28 reported the use of a sequential strategy of two-staged second-line tests comprising expert ultrasound and, if necessary, subsequent magnetic resonance imaging (MRI) for patients with RMI 1 cut-off values of 25 and < 1000, which achieved a sensitivity of 94% and specificity of 90%. The major advantage of the current strategy applying HistoScanning for patients with RMI 3 cut-off values of 105 and < 2100 is that similar accuracy is achieved without the need for patients to undergo an additional expertise-dependent, costly and time-consuming second-line test (e.g. MRI or computed tomography (CT)), as HistoScanning involves computerized analysis of data already acquired at the time of the initial ultrasound scan. The performance characteristics achieved using HistoScanning seem to be similar to those reported using a sequential strategy of initial triage with 10 simple rules, followed by subjective ultrasound assessment by an experienced sonographer for inconclusive cases falling outside the rules: sensitivity 91% (95% CI, 88 93%), and specificity 93% (95% CI, 91 94%) 11. The corresponding overall sensitivity and specificity of RMI alone were 68% (95% CI, 63 72%) and 93% (95% CI, 91 94%). There are other secondline tests with proven efficacy that are relatively easy to perform, for example color Doppler 30, but no Doppler data were available in this study. However, future research should evaluate the value of a sequential approach with RMI and HistoScanning that contains Doppler information and compare this with the simple-rules strategy. In conclusion, this study provides the first evidence of a potential role for ovarian HistoScanning in improving the performance of the RMI for the differential diagnosis of ovarian masses. The proposed strategy increases the proportion of patients appropriately referred to cancer centers managed by general gynecologists, and has the potential to reduce costs by avoiding additional expensive second-line tests such as MRI or CT. Future research should include comparisons with other models

7 Differentiating adnexal masses by RMI plus HistoScanning 97 in independent data sets and multicenter prospective clinical trials in a general population that incorporate cost-effectiveness/benefit analysis. ACKNOWLEDGMENTS We thank the investigators of the clinical trials: O. Lucidarme, J.P. Akakpo, B. Lauratet, J.P. Lefranc and P.H. Grenier (France); S. Grandberg and K. Schedvins (Sweden); M. Sideri, R. di Pace, D. Franchi, M. Bellomi and A. Maggioni (Italy); H. Levavi, R. Mashiach and I. Meizner (Israel); A. Schneider and J. Lange (Germany); and A.S. Absil, M. Solnick and A.R. Grivegnée (Belgium). Thanks to Mrs G. Fletcher and Mrs K. Ford (independent experienced ultrasonographers) for reviewing all archived 2D- and 3D-ultrasound images at University College, London. Financial support for E. Vaes: The research was partially supported by the General Direction of Technologies, of Research and of Energy of the Walloon Region [Convention 5117]. CONFLICTS OF INTEREST E. Vaes, R. Manchanda, P. Autier, A. Robert and U. Menon have no conflict of interest to disclose. R. Nir, D. Nir and H. Bleiberg are employees/consultants of AMD (Advanced Medical Diagnostics), the company that developed the HistoScanning technology, and are also stock owners of this company. REFERENCES 1. Cancer Research UK: Cancerstats. Ovarian cancer survival statistics. [Accessed 2010 Jul 1]; Available from: info.cancerresearchuk.org/cancerstats/types/ovary/survival/ 2. Le T, Krepart GV, Lotocki RJ, Heywood MS. Does debulking surgery improve survival in biologically aggressive ovarian carcinoma? Gynecol Oncol 1997; 67: Eisenkop SM, Spirtos NM, Montag TW, Nalick RH, Wang HJ. The impact of subspecialty training on the management of advanced ovarian cancer. Gynecol Oncol 1992; 47: Chan JK, Kapp DS, Shin JY, Husain A, Teng NN, Berek JS, Osann K, Leiserowitz GS, Cress RD, O Malley C. Influence of the gynecologic oncologist on the survival of ovarian cancer patients. Obstet Gynecol 2007; 109: Myers ER, Bastian LA, Havrilesky LJ, Kulasingam SL, Terplan MS, Cline KE, Gray RN, McCrory DC. Management of Adnexal Mass: Evidence Reports/Technology Assessments No Agency for healthcare Research and Quality: Rockville, MD, Timmerman D, Testa AC, Bourne T, Ferrazzi E, Ameye L, Konstantinovic ML, Van Calster B, Collins WP, Vergote I, Van Huffel S, Valentin L; International Ovarian Tumor Analysis Group. Logistic regression model to distinguish between the benign and malignant adnexal mass before surgery: a multicenter study by the International Ovarian Tumor Analysis Group. J Clin Oncol 2005; 23: Timmerman D, Van Calster B, Jurkovic D, Valentin L, Testa AC, Bernard JP, Van Holsbeke C, Van Huffel S, Vergote I, Bourne T. Inclusion of CA-125 does not improve mathematical models developed to distinguish between benign and malignant adnexal tumors. J Clin Oncol 2007; 25: Van Calster B, Timmerman D, Bourne T, Testa AC, Van Holsbeke C, Domali E, Jurkovic D, Neven P, Van Huffel S, Valentin L. Discrimination between benign and malignant adnexal masses by specialist ultrasound examination versus serum CA-125. J Natl Cancer Inst 2007; 99: Valentin L, Jurkovic D, Van Calster B, Testa A, Van Holsbeke C, Bourne T, Vergote I, Van Huffel S, Timmerman D. Adding a single CA 125 measurement to ultrasound imaging performed by an experienced examiner does not improve preoperative discrimination between benign and malignant adnexal masses. Ultrasound Obstet Gynecol 2009; 34: Valentin L, Hagen B, Tingulstad S, Eik-Nes S. Comparison of pattern recognition and logistic regression models for discrimination between benign and malignant pelvic masses: a prospective cross validation. Ultrasound Obstet Gynecol 2001; 18: Timmerman D, Ameye L, Fischerova D, Epstein E, Melis GB, Guerriero S, Van Holsbeke C, Savelli L, Fruscio R, Lissoni AA, Testa AC, Veldman J, Vergote I, Van Huffel S, Bourne T, Valentin L. Simple ultrasound rules to distinguish between benign and malignant adnexal masses before surgery: prospective validation by IOTA group. BMJ 2010; 341: c Jacobs I, Oram D, Fairbanks J, Turner J, Frost C, Grudzinskas JG. A risk of malignancy index incorporating CA 125, ultrasound and menopausal status for the accurate preoperative diagnosis of ovarian cancer. Br J Obstet Gynaecol 1990; 97: Bailey J, Tailor A, Naik R, Lopes A, Godfrey K, Hatem HM, Monaghan J. Risk of malignancy index for referral of ovarian cancer cases to a tertiary center: does it identify the correct cases? Int J Gynecol Cancer 2006; 16 (Suppl 1) Royal College of Obstetricians and Gynaecologists. Ovarian cysts in postmenopausal women, Evidence-Based Clinical Guideline No. 34; Available from: uk/womens-health/clinical-guidance/ovarian-cystspostmenopausal-women-green-top Tingulstad S, Hagen B, Skjeldestad FE, Halvorsen T, Nustad K, Onsrud M. The risk-of-malignancy index to evaluate potential ovarian cancers in local hospitals. Obstet Gynecol 1999; 93: Northern Cancer Network Gynaecological Oncology Service (NGOS). Northern Cancer Network Guidelines, Gynaecological cancers; [Accessed 2009 Jan 9]; Available from: Van Holsbeke C, Van Calster B, Valentin L, Testa AC, Ferrazzi E, Dimou I, Lu C, Moerman P, Van Huffel S, Vergote I, Timmerman D; International Ovarian Tumor Analysis Group. External validation of mathematical models to distinguish between benign and malignant adnexal tumors: a multicenter study by the International Ovarian Tumor Analysis Group. Clin Cancer Res 2007; 13: Geomini P, Kruitwagen R, Bremer GL, Cnossen J, Mol BW. The accuracy of risk scores in predicting ovarian malignancy: a systematic review. Obstet Gynecol 2009; 113: Tingulstad S, Hagen B, Skjeldestad FE, Onsrud M, Kiserud T, Halvorsen T, Nustad K. Evaluation of a risk of malignancy index based on serum CA125, ultrasound findings and menopausal status in the pre-operative diagnosis of pelvic masses. BrJObstetGynaecol1996; 103: Morgante G, la Marca A, Ditto A, De Leo V. Comparison of two malignancy risk indices based on serum CA125, ultrasound score and menopausal status in the diagnosis of ovarian masses. Br J Obstet Gynaecol 1999; 106: Lucidarme O, Akakpo JP, Granberg S, Sideri M, Levavi H, Schneider A, Autier P, Nir D, Bleiberg H; Ovarian HistoScanning Clinical Study Group. A new computer-aided diagnostic tool for non-invasive characterisation of malignant ovarian masses: results of a multicentre validation study. Eur Radiol 2010; 20:

8 98 Vaes et al. 22. Braeckman J, Autier P, Garbar C, Marichal MP, Soviany C, Nir R, Nir D, Michielsen D, Bleiberg H, Egevad L, Emberton M. Computer-aided ultrasonography (HistoScanning): a novel technology for locating and characterizing prostate cancer. BJU Int 2008; 101: Han J, Kamber M. Classification and Prediction, Data Mining: Concepts and Techniques. Elsevier: San Francisco, CA, 2003; DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 1988; 44: Aslam N, Tailor A, Lawton F, Carr J, Savvas M, Jurkovic D. Prospective evaluation of three different models for the preoperative diagnosis of ovarian cancer. BJOG 2000; 107: Manjunath AP, Pratapkumar, Sujatha K, Vani R. Comparison of three risk of malignancy indices in evaluation of pelvic masses. Gynecol Oncol 2001; 81: Timmerman D, Schwärzler P, Collins WP, Claerhout F, Coenen M, Amant F, Vergote I, Bourne TH. Subjective assessment of adnexal masses with the use of ultrasonography: an analysis of interobserver variability and experience. Ultrasound Obstet Gynecol 1999; 13: van Trappen PO, Rufford BD, Mills TD, Sohaib SA, Webb JA, Sahdev A, Carroll MJ, Britton KE, Reznek RH, Jacobs IJ. Differential diagnosis of adnexal masses: risk of malignancy index, ultrasonography, magnetic resonance imaging, and radioimmunoscintigraphy. Int J Gynecol Cancer 2007; 17: Guerriero S, Alcazar JL, Pascual MA, Ajossa S, Gerada M, Bargellini R, Virgilio B, Melis GB. Intraobserver and interobserver agreement of grayscale typical ultrasonographic patterns for the diagnosis of ovarian cancer. Ultrasound Med Biol 2008; 34: Medeiros LR, Rosa DD, da Rosa MI, Bozzetti MC. Accuracy of ultrasonography with color Doppler in ovarian tumor: a systematic quantitative review. Int J Gynecol Cancer 2009; 19: