Observer Variation in 2-Dimentional Ultrasound Imaging for Prediction of Endometrial Cancer in Women with Post- Menopausal Bleeding

Transcription

1 Observer Variation in 2-Dimentional Ultrasound Imaging for Prediction of Endometrial Cancer in Women with Post- Menopausal Bleeding Research Year Report Stud.med. Stine Rydbjerg Department of Gynecology and Obstetrics Aarhus University Hospital, Skejby

2 CONTENTS Acknowledgements Abstract Background Endometrial Cancer Diagnosing Endometrial Cancer Stage Determination of endometrial cancer Ultrasonography Standardized terminology Observer variation Diagnostic set-up Aim Methods and Materials Patients and design Examiners Ultrasonography Endometrial sampling Hysteroscopy Pathology Statistical Methods Results Basic information on the study population Observer Variation Diagnostic performance of the office endometrial biopsy Diagnostic performance of ultrasonography Diagnostic performance of hysteroscopy Comparisons of the diagnostic tests Discussion Terms, standardization and pattern recognition Observer Variation Diagnostic performance of the office endometrial biopsy Diagnostic performance of ultrasonography Diagnostic performance of hysteroscopy Comparisons of the diagnostic tests Page of 40

3 Strengths and limitations Summary Perspectives Reference List Page of 40

4 ACKNOWLEDGEMENTS This study was carried out at the Department of Obstetrics and Gynecology, Aarhus University Hospital, Skejby between the 1 st of February 2011 and the 31 of January During this period many people have supported me and I owe each and all of them my deepest gratitude. I would like to give special thanks to my principal supervisor Margit Dueholm for her invaluable help throughout the project. I am grateful for her patience, encouragement and guidance during the writing process, and for the help and support I got to understand the complex world of statistics and Stata. I would also like to thank Julie Wulf Holm, Charlotte Møller and Gitte Ørtoft for their help with acquiring the scans as well as all the staff at the Gynecological Outpatient Clinic and the surgical Outpatient Clinic at Aarhus University Hospital, Skejby. Without their patience and willingness to let me perform the scans even when the time schedule was running very late, this project would not have succeeded. I also thank Estrid Stæhr Hansen from the Department of Pathology for her important assistance with pathologic analysis. I am very thankful for the financial support from Kræftens Bekæmpelse, which made this research project possible. A special thank is owed to the staff and students at A/Y research unit for the great and warm atmosphere. Elise Klæstrup, Sara Bastholm, Eva Mikkelsen and Mathias Gottschalk Heide I thank for their support with my research and for never giving up on me in their attempts to socialize. Thank you for making the days in the office so great, for answering questions, and sharing experiences in research as well as in life. Last but not least I owe a great thank to my husband and three children who never walked out on me, even though this project at times took up all of my time. Thank you for letting me have the opportunity to explore the world of research. Page of 40

5 ABSTRACT AIM: The aim of this study was to assess the applicability of a set-up of standardized terms for pattern recognition in ultrasonography for detection of malignant pathology in the postmenopausal endometrium. We wanted to determine the diagnostic efficiency of this set up in the hands of inexperienced examiners as well as determine observer variation between experienced and inexperienced examiners. Subsequently we wanted to compare observer variations found with trans-vaginal ultrasonography (TVS) with observer variations found with TVS with gel contrast (GIS). Different standardized terms for TVS as well as GIS were evaluated in order to assess the effect of these methods on observer variation. Finally the diagnostic efficiency of evaluation with TVS/GIS, endometrial sampling and hysteroscopy was compared. METHODS: Fifty-three women with postmenopausal vaginal bleeding were independently and blinded evaluated with TVS and GIS. All women were scanned by an experienced gynecologist and a junior research assistant. The endometrial morphology was evaluated with regards to standardized parameters and the inter-observer variability was calculated. Subsequently, endometrial office biopsies were sampled and a hysteroscopy was performed. The golden standard, to which the endometrial biopsies, ultrasonographic results and hysteroscopic visual evaluation were compared, was the histo-pathological results of the resectoscopic biopsies taken during hysteroscopy. RESULTS: More than ¾ of endometrial cancers were identified with TVS by the experienced examiner and with GIS by both the experienced and the inexperienced examiner. Although observer variations between were considerable for most of the standardized terms there was a good agreement on the different diagnosis, especially when applying GIS. For each standardized parameter there was no significant difference in the disagreement between the two examiners at GIS vs. TVS. However, for the total number of agreements or disagreements of diagnosis, agreements in with GIS were statistically higher than with TVS. In the present set-up no significant differences were found in distinguishing between malignant and benign endometrium using office biopsy, TVS, GIS or hysteroscopy. CONCLUSION: GIS seems to increase reproducibility in TVS-results when evaluating inter-observer variability between experienced and inexperienced examiners. However, larger scale studies are needed to confirm this finding. Findings in the present study suggest that a combination of various standardized terms and the fact that the examiner is forced to approach the uterus systematically can improve the diagnostic performance, especially for inexperienced examiners. In conclusion, applicability of pattern recognition in ultrasonography for diagnosis of endometrial pathology seems promising. Page of 40

6 BACKGROUND ENDOMETRIAL CANCER Endometrial cancer is the second most common gynaecological cancer in Denmark after cancer ovarii. It affects approximately 600 women each year. Endometrial cancer is rare among women below 45 years of age and most common among women above 55 years. The risk is highest amongst postmenopausal women with bleeding and above 60 years of age. Factors that predispose to endometrial cancer include high BMI, PCOS, Selective Estrogen Receptor Modulators and sequential treatment with oestrogen-gestagen, all factors that contribute to a continuous high level of estrogen in the endometrium [1]. Cancers found in FIGO stadium I has a good prognosis with a post-operative survival rate that does not differ essentially from the background population [1]. The 3-year survival rate of cancers found in FIGO state I is 89.6% versus 23.0% in state IV [2]. In Denmark there has been a substantial delay from a woman is referred form the general practitioner until the treatment is initiated. 10% of endometrial cancers in Denmark experience such a delay [3]. This is considered to be due to an inadequate diagnostic set up either because the diagnostic methods are too vague or because of insufficient compliance in the clinical routine. It is therefore of great importance to improve the diagnostic set up for quick and safe evaluation of women with postmenopausal bleeding. DIAGNOSING ENDOMETRIAL CANCER Meta-analysis has shown that endometrial thickness < 5 mm and a well-defined endometrial-myometrial border evaluated with 2 dimensional trans vaginal ultrasonography (2D TVS) excludes endometrial cancer in postmenopausal women with bleeding with most certainty [4], even in relatively inexperienced hands [5]. In large studies of women with postmenopausal bleeding one out of five women with endometrial thickness of more than 5 mm had endometrial cancer [6]. The Danish Gynecological Cancer Group (DGCG) has created uniform guidelines for diagnosis of endometrial cancer. Diagnosis is made based upon trans-vaginal ultrasonography (TVS) and, in case of endometrial thickness > 4 mm, endometrial histo-pathology is evaluated from office endometrial biopsies. DGCG recommends women with postmenopausal bleeding and endometrial thickness < 4 mm to undergo a reevaluation in case of repeated bleeding after a period of 3 months. In Denmark the Danish Central office of health has recommended fast track diagnostic pathways for postmenopausal women with vaginal bleeding. The standard recommendations are based on guidelines from DGCG. Women with a malignant histo-pathologic evaluation are referred to gynecologic oncologic centers as the one in which the present study is carried out. The histo-pathological evaluation is at hand after one week and a malignant response ensures an immediate referral for stage determination and adequate treatment. However, sampling of the biopsy is a relatively painful procedure and the failure rate in sampling is as high as 19% [7]. Furthermore, endometrial samples obtained by office suction biopsies often contain insufficient material and endometrial cancers may be missed [8]. Page of 40

7 The gold standard for evaluation of endometrial pathology as well as treatment of benign endometrial pathology is hysteroscopy with biopsy. According to guidelines from Danish Society Obstetrics and Gynecology (DSOG), women with endometrial thickness > 4 mm and benign histo-pathological evaluation and women where the biopsy was insufficient should, based on these facts, be offered hysteroscopy or Saline Infusion Sonography (SIS) [9]. Hysteroscopic removal of all localized pathology seen at SIS is recommended. STAGE DETERMINATION OF ENDOMETRIAL CANCER To offer minimal invasive hysterectomy preoperative stage determination in endometrial cancer is needed. MRI (Magnetic Resonance Imaging) or TVS are the most optimal imaging methods for evaluation of depth of myometrial invasion. Moreover, combining MRI or TVS with hysteroscopy and resectoscopic biopsies may increase the diagnostic performance in preoperative evaluation of cancer stage determination [10]. Unfortunately hysteroscopic biopsies or removal of focal pathology prior to MRI or ultrasound makes assessment of myometrial ingrowths of endometrial cancer difficult by these imaging methods. In suspected cases of endometrial cancer stage determination with MRI or TVS should therefore be performed before hysteroscopy. ULTRASONOGRAPHY In short, the principle of ultrasonography is that high frequency sound waves (ultrasound) are confined to a narrow beam that may be transmitted through or refracted, absorbed or reflected by the tissue towards which they are directed. Depending on the tissue density some sound waves are reflected back to the transducer. The echo of these reflections is converted into electrical impulses that are subsequently presented as a picture of the tissue the transducer is directed against. Ultrasound is the cheapest image technique for picturing uterus. Today it is at hand in every gynecological office and TVS is normally performed as part of the general gynecological examination. Evaluation of the traditional TVS takes place while scanning and it is inappropriate for evaluation of still images. Rethinking of the traditional examination with 2D TVS includes contrast ultrasonography, which improves the imaging of the endometrial surface and thereby allows for evaluation of focal abnormalities, which is often seen in endometrial cancer [11, 12]. In 3 dimensional (3D) image techniques the use of gel infusion sonography (GIS) instead of saline for infusion (SIS) ensures a stable picture without motion artifacts because gel in contrast to saline remains for a while in the uterine cavity. Furthermore, modern scanners can obtain and store 3D images, which can be sent electronically to experienced assessors elsewhere. Ultrasonography requires competence in image optimization and in recognition of pathology and is thus very observer dependent. Modern scanners and software have made image optimization relatively easy, while recognition of pathology still requires routine and experience. Simplifying recognition of pathology to recognition of a certain pattern and dividing the assessment of the uterine cavity into predetermined variables might uniform the evaluation and could be a solution to the question of how to reduce the observer variation. Page of 40

8 Moreover, it may simplify the learning process. In a set-up with a sufficient volume of repetitive scans a standardized evaluation of endometrial pathology may improve learning of this specific task. In 2007 Opolskiene G. et al. determined which endometrial morphology characteristics assessed by grey-scale ultrasonography were useful for discriminating between benign and malignant endometrium in women with postmenopausal bleeding and endometrial thickness 4.5 mm [13]. They found the best ultrasonography variables to predict malignancy were heterogeneous endometrial echogenicity and endometrial thickness. Furthermore, irregularity of the endometrial-myometrial border and the internal structure of the endometrium, i.e. cystic, hyperhypo- or isogenic endometrial echogenicity, were found as predictors of different endometrial pathology. Subsequently, a set of standardized parameters was suggested by the IETA group in 2008 [14]. The advantage in TVS compared to office biopsy as diagnostic tool could be the lower failure rate. The proportion of cases in which the endometrium cannot be measured may be about 10% at SIS [15] in contrary to the office biopsy sampling failure rate that has been found as high as 19%. Moreover, the time frame is a great advantage with TVS, where the tentative diagnosis is made right away whereas in the office biopsy the women must wait a week for the diagnosis. STANDARDIZED TERMINOLOGY Standardized terms used for assessing the uterine cavity were suggested by the International Endometrial Tumor Analysis (IETA) group [14]. IETA made the consensus statement on terms, definitions and measurements to describe the sonographic features of the endometrium and uterine cavity after forming the group at the World Congress of Ultrasound in Obstetrics and Gynecology in At present the relationship between ultrasonographic features and presence or absence of pathology is still not known. The consensus statement was meant to streamline definitions and terms used to describe the ultrasonographic findings for prospective studies. A number of studies have subsequently been made, including logistic regression models for calculating individual risk of endometrial malignancy [13, 16] The International Ovarian Tumor Assessment (IOTA) group has worked out similar terms for assessing ovarian pathology [17]. Using these standardized terms has shown a high diagnostic efficiency in studies on ovarian pathology [18]. Whether the same types of standardized terms are applicable when assessing the uterine cavity is not known until more extensive studies has been formed. The different variables suggested by the IETA group can be used for evaluating the endometrium after scanning each woman. This approach may have a high diagnostic efficiency for diagnosis of malignant endometrium. OBSERVER VARIATION The sensitivity of TVS for diagnosing endometrial cancer could be comparable to that of the routinely taken office curettage, if performed by an experienced gynaecologist [4, 19, 20]. However, ultrasonography is known to be highly observer dependent. A study from 2001 has shown a kappa value of 0.68 ( ) in inter-observer agreement in evaluation of the normal/abnormal uterine cavity in pre-menopausal women. In that study the observers were almost Page of 40

9 equally experienced and the inconclusive examinations (poor visualization and/or difficult interpretation) were excluded [21]. Another study by Alcazar, JL. et al. shows a good inter-observer agreement for evaluating the vessel pattern in the endometrium assessed with power Doppler when comparing two expert-evaluations, but when compared to less experienced examiners the agreement decreased [22]. To our knowledge a study assessing inter-observer agreement between an expert and a person with no experience in ultrasonography has not yet been performed until this study. DIAGNOSTIC SET-UP Experienced sonographers have a high efficiency for diagnosis of endometrial cancer in evaluation of endometrial pathology based on ultrasonography and patients characteristics [12, 16, 23]. This study is part of a larger study, with the aim to evaluate and compare the diagnostic accuracy of endometrial sampling, hysteroscopy, MRI, 2D and 3D TVS and GIS with and without Doppler flow for assessment of endometrial pathology in postmenopausal women. Based on ultrasound criteria and patient characteristics we might be able to identify a risk group for endometrial cancer, and propose a set-up for an effective and fast diagnosis and stage determination of endometrial cancer. It is essential to create a generally applicable diagnostic set-up, which provides an early and effective diagnostic procedure. To do so it is of great importance to know which resources are at hand to create a time frame for diagnosis that is shorter than the existing time frame and at least as effective. If less experienced doctors can distinguish a malignant endometrium from a benign, much faster diagnostic procedures can be implemented. In gynecologic praxis ultrasonography is performed by the common gynecologist as a part of the office evaluation. Thus ultrasonographic patterns of endometrial pathology are evaluated by doctors without special competence in ultrasonography. Even though observer variation between experienced doctors is low it may not be the case for inexperienced examiners. The application of new terms in general practice require that they are so uniformly defined, that even inexperienced examiners may learn to use them after a short period of training. The applicability of the present set up depends on whether or not the diagnostic efficiency in the small group of patients was acceptable and in line with present standard. Page of 40

10 AIM The aim of the present study was to evaluate the applicability of a set-up of standardized terms for pattern recognition in ultrasonography for detection of malignant pathology in the postmenopausal endometrium in the hands of inexperienced examiners after a short training period. Different standardized terms for conventional trans-vaginal ultrasonography (TVS) and TVS after installation of gel in the uterine cavity (GIS) was evaluated to a. Determine the observer-variation in the different parameters between experienced and inexperienced examiners. b. Comparing these observer variations seen with TVS and with GIS to evaluate the effect of these methods on observer variation. c. Determine the diagnostic efficiency of this set-up to identify endometrial pathology in the hand of inexperienced observers. d. Compare the diagnostic efficiency of the evaluation with endometrial sampling and hysteroscopy. Page of 40

11 METHODS AND MATERIALS PATIENTS AND DESIGN The study population consists of 53 of 141 consecutive postmenopausal women with bleeding, scheduled for either hysteroscopy or trans-vaginal ultrasonography (TVS) in the period to The women were referred to Aarhus University hospital, Skejby, which is one of five reference centers for endometrial cancer in Denmark. 27 (51%) women were referred from a general practitioner. 16 (30%) came from a specialist in gynecology and 10 (19%) from a county hospital with less experience for further assessment. The women were considered postmenopausal if they reported absence of menstruation for at least one year provided that the amenorrhea was not due to disease, medication or pregnancy. Postmenopausal bleeding was defined as any vaginal bleeding in a woman not on hormone replacement therapy (HRT) or unscheduled bleeding in a woman on HRT. All women with postmenopausal bleeding and an endometrial thickness >5 mm measured with trans-vaginal ultrasonography (TVS), using the double layer technique, were included. In all cases a gynecologic examination was performed according to the standard in the department at first visit and a cervical smear was made when a smear made within a year was not at hand. All patients were examined with TVS and the endometrial thickness was measured. Women with an endometrial thickness <5 mm did not have any further examinations and were not included in this study as their risk of endometrial cancer is minimal [8, 24]. They were all referred to be reevaluated in case of re-bleeding after a period of three months. In all cases of women with an endometrial thickness 5mm an endometrial office suctions biopsy was sampled. They were scheduled for a second visit after approximately one week to receive the histo-pathological result from the office biopsy. At the second visit they were scanned according to the protocol by one of two (MD, CM) experienced examiners with more than 10 years of experience in TVS. In order to blind the examiner, the results of the biopsies were placed in a closed envelope, and first opened and given to the patient after scanning and completing the standardized form. In the cases where an endometrial office biopsy could not be collected at the first visit, as well as in the cases where women were referred from other hospitals or from a specialist in gynecology, scanning was performed immediately before hysteroscopy. Patients that were included with TVS and GIS according to protocol immediately before hysteroscopy where a) Patients where an endometrial sampling could not be performed at office settings, b) patients referred from specialist in gynecology for further hysteroscopy often because of benign pathology and c) patients with endometrial cancer or hyperplasia with atypia referred for hysteroscopic stage determination. Patients with a prior resectoscopic removal of endometrial tumor, prior evaluation of cervical cancer, stage 3-4 endometrial cancer, serous carcinoma, clearcell carcinoma, sarcomas and carcino-sarcomas were not scheduled for hysteroscopy. Moreover, patients with endometrial cancer had no hysteroscopic staging when advanced age or serious co morbidity indicated less extensive surgery independent of the stage of endometrial cancer. Page of 40

12 In all cases of scanning before hysteroscopy, the procedure was marked post-ul and the examiners (MD, CM) of TVS and GIS were blinded to records and prior endometrial samples. After completing the ultrasonographic examination and the standard form, hysteroscopy was performed by the same examiners as performed TVS and GIS. In total 141 women were evaluated by TVS, GIS and Hysteroscopy in the study period where two examiners performed examinations. 53 women were evaluated with TVS by both an experienced gynecologist (MD, CM, GØ) and a junior research assistant (SR, JW). These were the women included in the present study. The tight time schedule in the outpatient clinic hindered that all 141 women were evaluated by both examiners. The women who underwent evaluation by both observers were randomly chosen according to the time at hand, hence, no systematic selection upon certain criteria was made. They account for a representative section of the study population as a whole. The present study was part of a larger study running from to where the diagnostic efficiency of TVS and GIS was evaluated by experienced examiners. During this period a total of 176 examinations were performed. In the total period only 27 (6.5%) of women who met the inclusion criteria were not scanned according to protocol, which were primarily due to logistic problems. EXAMINERS All 53 examinations were performed by both an experienced gynecologist (MD (42), CM (9), GØ (2)) and a junior research assistant with a minimum of ultrasound experience (JW (18), SR (35)). Two inexperienced observers participated in the study in order to secure the level of inexperience. The first observer JW had prior experience in evaluation of 3D volumes for staging endometrial cancer in a separate study of her own and had attended approximately 75 TVS examinations in patients with endometrial cancer. Subsequently she had a short introduction with 7 days of supervised scans and then performed the examinations independently. The second inexperienced observer had an introduction of two months with 14 days of supervised scans, including education in evaluation of the uterine cavity in the outpatient clinic. When possible the observer with most experience (MD) performed the scans. At first, conventional gray-scale 2D and 3D TVS examinations were performed by both observers. The two observers performed TVS in turns without knowledge of the other performer's results, evaluated the examinations and filled the form. Subsequently all patients were scanned after gel infusion into the uterine cavity. Insertion of the catheter was always performed by the experienced investigator. The experienced gynecologist further recorded 2D and 3D power Doppler images. Page of 40

13 ULTRASONOGRAPHY TVS was carried out using an E8 ultrasound machine (GE Healthcare, 300 N. Grandview Blvd., Waukesha, WI 53188, USA) equipped with a high frequent trans-vaginal transducer. A pre-installed standard program was used scanning the women and each examiner independently optimized the images on the scanner. 2D images were recorded scanning in transversal plane from cervix to fundus and sagittal plane and from cornu to cornu with and without saline contrast. In all cases the examinations were videotaped and 2D and 3D pictures were stored for later evaluation. The analysis of the gray-scale 2D ultrasonography on endometrial morphology included visual evaluation of the following: Endometrial thickness measured by double layer technique [14]. Internal endometrial echo structure (hyperechogenic, hypoechogenic, isoechogenic, cystic, impossible to evaluate). Homogeneity of the endometrial echogenicity (homogeneous, heterogeneous, impossible to evaluate). Presence of a bright line separating the endometrial echo from the myometrium (visible, intact, not visible). Abruption of the endometrial-myometrial border. Regularity of the endometrial-myometrial echo (regular, irregular, impossible to evaluate). After TVS GIS was performed. A small flexible sterile catheter (baby-feeding tube Vigor , Belgium) mounted with a 10-mL syringe with Instillagel (Farco Pharma) was introduced into the uterine cavity. During instillation of gel the pressure was manually adjusted until it was sufficient for expansion of the uterine cavity. Concomitantly, the distension was observed by TVS and continued until the entire uterine cavity was clearly visible. The uterine cavity was evaluated in sagittal and coronal views and pictures were taken for documentation. The analyses were further improved with evaluation at GIS where the following parameters were evaluated: Structure of the endometrial surface (smooth, polyploidy, irregular) Is there a local or a non-local lesion (i.e. a lesion < or > 25% of the surface). Structure of the surface of the local lesion. Analysis of the power Doppler ultrasonography included visual evaluation of the following (only performed by the experienced gynecologist. This extra advantage has to be taken into account when considering the expected higher diagnostic accuracy for the experienced examiner): Dominant or non-dominant vessels (single or double). Focal or multifocal origin of the vessels. Numbers of vessels (few or many). Regularity of vessel branching (regular or non-regular). Size of blood vessels. Page of 40

14 Presence of color splashes. Presence of areas with densely packed vessels. Presence of circular flow. Based on the standard evaluation of TVS, the examiner coded a diagnosis of normal/proliferative endometrium, benign endometrial polyp, submucosus fibroma, hyperplasia or cancer. The examiner was allowed to give three diagnoses with priority. The code hyperplasia could account for either hyperplasia, complex hyperplasia or hyperplasia with atypia. Before performing the statistical calculations all codes were checked for irregular coding and redefined as benign or malignant. Benign codes were normal/proliferative endometrium, benign endometrial polyp, submucosus fibroma, hyperplasia and complex hyperplasia. Malignant codes were hyperplasia with atypia and cancer. ENDOMETRIAL SAMPLING Office endometrial samples were collected by different gynecologists. Samples taken in the University out-patient clinic were collected by one of the experienced gynecologists using a MedGyn endosampler, 3 mm. Med 10 ml. syringe (328N. Eisenhower Lane. Lombard IL , USA). Endometrial samples taken in the University Clinic were evaluated by one of two experienced onco-pathologists, while samples taken at other Hospitals were evaluated by the local pathologist. HYSTEROSCOPY Hysteroscopic evaluation with respect to the pathology was performed according to standard forms, and a diagnosis was made based on final findings at hysteroscopy. Hysteroscopy was performed in the day surgery clinic under a short general anesthesia by one of five experienced gynecologists. When TVS and GIS were performed immediately before hysteroscopy it was carried out by one of the two experienced investigators (MD, CM). Thus the hysteroscopic examination was not blinded to the findings on ultrasonography, and so the evaluation of the hysteroscopic findings should be considered a combination of ultrasonography and hysteroscopy. With the patient under general anesthesia, the cervical canal was dilated to Hegar 10.5, and a Storch resectoscope (Storch, N. Westport, CT, USA, 26 Fr12 degree oblique optic) was inserted. The uterus was distended with Purisol and the pressure was maintained at 60-80mm Hg. The presence of localized or diffuse lesions was described as suspect findings of hyperplasia, malignancy, benign polyps or myomas and was recorded on a standard form, where the diagnosed based on hysteroscopy was given. In suspect malignancy subjective assessment of cervical involvement was noted, when tumor boarders reached the internal cervical ostium. PATHOLOGY During the hysteroscopy all focal changes were removed, and in cases of large diffuse changes resectoscopic biopsies were taken. Biopsies were taken from the area of the endometrium with the maximum changes. Three to five biopsies were sampled. Page of 40

15 In cases of normal hysteroscopic findings one biopsy was taken from the front wall of the uterine cavity, one from the back wall of the uterine cavity and curettage was performed. In cases that were suspect for cancer biopsies from the cervix were taken as well, to identify cervical involvement. In some cases where the women underwent hysterectomy, it later emerged that the focal malignant area in the uterine cavity had been radically removed so that no detectable malignant cells were left in the uterus. The contrary has not occurred in our material. In one single case the histo-pathological diagnose of the resectoscopic biopsies showed complex atypical hyperplasia and subsequently cancer in the evaluation of the uterus after hysterectomy. Due to the fact that these two conditions both result in hysterectomy the resectoscopic biopsies will in the following account for the golden standard whether or not the woman later had a hysterectomy. The histo-pathological evaluation was carried out by one of two experienced onco-gynecological specialist and findings were used as a gold standard to evaluate the diagnostic efficiency of TVS and GIS as well as the endometrial biopsy taken in the outpatient clinic on the first visit and hysteroscopic visual evaluation. The final pathologic evaluation was categorized as insufficient, atrophic endometrium, normal/proliferative endometrium, benign endometrial polyps, submucosus fibroma, low grade hyperplasia without atypia, high grade hyperplasia without atypia, high grade hyperplasia with atypia, cancer and other changes. In ultrasonography it is not possible to discriminate between hyperplasia with atypia and cancer. As these two conditions both result in hysterectomy they were categorized as malignant conditions in the final statistical analysis where the three priories of diagnosis were added up and a new variable (malignanl/benign) created. STATISTICAL METHODS Statistical analyses were undertaken using the Stata/IC 11.2 package (StataCorp LP, College Station, Texas, USA, release 12). Mean (CI 95%) was given for continuous normal distributed data. When comparing patients with and without malignant disease with respect to continuous distributed parameters like age and BMI, Students t-test was used to calculate the difference in mean. Reproducibility was described by comparing the results from two examiners on the same patient group. The pair wise comparison reflects the difference in diagnosis between the two independent examiners with regards to the interobserver variation, which also reflects how reproducible a certain test or method is. The agreement in diagnosis was calculated using Cohen s kappa coefficient. Kappa =p0-pc /1-pc is the observed agreement corrected for the expected agreement by chance. Kappa is defined as the difference between observed and expected agreement (by chance) expressed as a fraction of the maximum difference (perfect agreement and chance agreement). Kappa can vary between 1 and +1. Page of 40

16 Two observers may either agree or disagree as illustrated below. Observer II Observer I + disease - disease Total + disease a b a+b - disease c d c+d Total a+c b+d n The observed agreement p0 is a+d/n. The expected agreement by chance pc is the expected agreement on positive finding added to the expected agreement on negative findings [(a+c)/n x (a+b)/n] + [(b+d)/n x (c+d)/n]. A kappa of less than 0.20 indicated poor agreement, fair agreement, moderate agreement, good agreement and very good agreement. The Wilcoxon signed rank test was used to examine the median of difference between the paired observations under the null hypothesis that the median of the paired differences is zero. Two-tailed P < 0.05 was considered statistically significant. Bland-Altmann plots were used to estimate observer variation in measurements of the endometrium assessed by TVS as well as GIS. Histograms and Q-norm plots of measurements as well differences in measurements were made to test normal distribution. Endometrial thickness measurement results from the two observers were plotted against each other. If measurements clustered around the diagonal line of agreement there was a high correlation. Similarly mean was plotted against difference between each paired measurement around a line of zero difference. If plots clustered around the line, differences could be assumed to be normally distributed without any systematic change by increasing endometrial thickness. 2x2 tables were used to calculate sensitivity, specificity and positive and negative predictive values (PPV and NPV) in office biopsy, hysteroscopy, TVS and GIS for each examiner respectively. As golden standard, resectoscopic biopsies taken during hysteroscopy were used. Sensitivity evaluates the tests ability to predict malignancy defined as the proportion of true positives correctly identified as such. Specificity evaluates the tests ability to detect the truly healthy women, i.e. the proportion of true negatives correctly identified as such. Sensitivity and specificity are characteristics of the test. Their values are not dependent on the prevalence of disease in the population and thus they are important in assessing tests that shall be applicable to another study population. Page of 40

17 PPV is the percentage of truly positive amongst the positive test results and NPV is the truly negative amongst negative test results. The predictive values are highly dependent of the prevalence of endometrial cancer in the test population. The predictive value of a positive test is highest when the prevalence of disease is high in the tested population and subsequently the predictive value of a negative test is highest when the prevalence is low. As an example a 2x2 table is shown below. Test Benign Malignant Total Benign Malignant Total A B (A+B) (truely negative) (false negative) (tested negative) C D (C+D) (false positive) (truely positive) (tested positive) (A+C) (gold-standard negative) Gold-standard (B+D) (gold-standard positive) N -Prevalence: (B+D)/N 100 -Sensitivity: D/(B+D) 100 -Specificity: A/(A+C) 100 -PPV: D/(C+D) 100 -NPV: A/(A+B) % CI of sensitivity, specificity, PPV and NPV was calculated. McNemars chi 2 test was used to compare diagnostic methods as well as diagnostic efficiency between observers. The test is based on the numbers of discordant pairs i.e. it tests only the cases in which the examiners or methods disagree. To use this test new variables had to be created i.e. correct and incorrect diagnosis. An example is displayed below: Experienced Inexperienced Correct Incorrect Total Correct a b a+b Incorrect c d c+d Total a+c b+d n The null hypothesis of McNemar is, that a+c = a+b and b+d =c+d, that is, that the number of correct findings by the experienced is the same at the inexperienced and the same for not correct. Page of 40

18 Thus the null hypothesis is b=c And The test determines a chi 2 value and a p-value. Two-tailed P < 0.05 was considered statistically significant. Page of 40

19 RESULTS BASIC INFORMATION ON THE STUDY POPULATION Of the 53 women included 32 (60.0%) had benign and 21 (40.0%) had malignant endometrium. Mean age was 63 years and body mass index (BMI) 31. Histo-pathological diagnoses are shown in Table 1. Table 2 shows patients characteristic for women with benign and malignant endometrium. Women with malignant endometrium were older than those with benign endometrium (mean 67.8 ( ) years vs. median 60,36 (57.21 ; 63.51) years; P=0,0000) and had a higher BMI (median 33,49 (30,53 ; 36,46) kg/m 2 vs. 29,21 (26.89 ; 31.53) kg/m 2 ; P=0,0007). Women with malignant endometrium had a significantly thicker endometrium than those with benign in the cases measured by both the experienced and inexperienced examiner with TVS. Table 1: Histo-pathological findings in the 53 women Findings n (single diagnosis) n (%) Benign (n = 32) Polyp * Athrophic endometrium 4 13 Hormone-influenced endometrium Hyperplasia Simple 2 Complex Malignant (n = 21) Hyperplasia with atypia 2 3 ǂǂ Adenocarcinoma Total (n = 53) *Six had atrophic endometrium, two had hormone-influenced endometrium, and one had fibroma, one had atrophy and complex hyperplasia, one had atypia and three had cancer as well as a polyp. Six had polyps, one had polyp and complex hyperplasia, one had complex hyperplasia, one had simple hyperplasia as well as atrophy. Both had polyps as well. One had myoma and one had simple hyperplasia as well. ǂOne had polyp and atrophy, one had complex hyperplasia and one had cancer as well. ǂǂOne had a polyp as well. Four had polyps and one had complex hyperplasia as well. 2 3 ǂ With TVS there were a total of 4 and 6 inconclusive examinations for experienced and inexperienced examiners respectively (3 of which were the same patients). With GIS there were 4 and 7 inconclusive examinations respectively (2 of which were the same patients). Page of 40

20 In this small study population there seemed to be no connection between examiners giving incorrect diagnosis and bad/inconclusive examinations, thus it would not influence the results e.g. lower the observer variation excluding women with bad/inconclusive evaluations. Table 2: Characteristics of the 53 women Characteristics Total (n=53) Benign (n=32) Malignant (n=21) Difference Mean (95% CI) Mean (95% CI) Mean (95% CI) (p) N(%) N(%) N(%) N(%) Age (years) 63 (61-66) 60 (57-64) 68 (63-72) 7.4 (0.0057) BMI (kg/m 2 ) 31 (29-33) 29 (27-32) 33 (31-36) 4.3(0.0219) Nulliparious 10 (19.6%) 6 (19.4%) 4 (20.0%) Hypertension 28 (53.9%) 17(53.1%) 11(55.0%) Diabetes? 6 (11.8%) 5 (15.6%) 1 (5.3%) Years in menopauses?? 11.5 ( ) 8.8 ( ) 16.1 ( ) 7.3 (0.0211) Endometrial thickness (mm) Experienced 13.2 ( ) 9.9 ( ) 18.1 ( ) 8.2 (0.0002) Inexperienced 14.8 ( ) 12.4 ( ) 18.4 ( ) 6.0 (0.0045) Quality of examinations: -TVS: exp/inexp* Good 12/8 6/8 Intermediate 17/20 13/10 Bad 2/4 2/1 Inconclusive 0/0 0/1 -GIS: exp/inexp** Good 11/16 7/5 Intermediate 10/14 7/8 Bad 2/5 2/2 Inconclusive 0/0 0/0 Data was analyzes as two independent samples from normal distributions based on the students t-test. The p-value reflects the significance of difference between the benign and malignant. Data are given as mean (95% CI) or n (%). Information lacking in one patient in the benign group. Information lacking in two patients, one in each group. Information lacking in one patient in the cancer group. ǂ Information lacking in two patients in the cancer group. ǂǂ Information lacking in three patients in the cancer group. *Examinations were labeled according to quality of imaging as good, intermediate, bad or inconclusive. In the following we consider bad and inconclusive as inconclusive. Information lacking in one patient in the cancer group for inexperienced and one in the benign group for experienced. **Out of 49 scans by the experienced examiner information is lacking in three patients in the cancer group and five in the benign Out of 45 scans by the inexperienced examiner information is lacking in two patients in the cancer group and two in the benign Page of 40

21 OBSERVER VARIATION Table 3 shows the diagnostic agreement between experienced and inexperienced examiner of the various ultrasonographic terms. Cohen s kappa was calculated from a series of the standardized terms, which were evaluated after scanning each woman. The most reproducible terms were the ones with the highest kappa, that is, the ones with the smallest observer variation. In 53 patients the evaluation of the endometrium by the different standardized terms was performed by both the experienced and the inexperienced observers. We allowed the observers to let the terms be not evaluable. Women, whom were considered to have not evaluable terms by one of the observers, were excluded from the analysis of observer variation, thus the number of patients varies in various terms. Most endometrial echoes (88.7%) turned out to be hyper-echogenic with only (11.3%) of the patients with hypo- or isoechogenic evaluated by the experienced observer. Thus a very high observer agreement is expected and few disagreements influence much on the kappa value. In this case the agreement (78.4%) underscores the expected agreement (80.74%) and the kappa is therefore negative (-0.12). This parameter is not shown in the table. Moreover, we evaluated an intact or ruptured endometrial-myometrial halo. Observer variation was not calculated on this parameter as the endometrial-myometrial halo was only identified by both examiners in 30 patients. The endometrial structure (homogeneity, echogenicity) could be evaluated by the experienced observer in all 53 patients. Only in two patients the inexperienced observers could not evaluate the endometrial structure. The kappa values for the terms homogeneity (0.49) and cystic endometrium (0.53) had moderate agreement between the observers. The endometrial-myometrial boarder could not be evaluated in five cases by neither the inexperienced nor the experienced, or both with regards to homogeneity, regularity and bright line. When assessing the variable abrupt another nine cases were excluded leaving 48 and 39 cases for analysis respectively. The evaluation of the endometrial-myometrial boarders seemed to be very observer dependent i.e. less reproducible. There was fair agreement on the terms homogeneity of the boarder (0.21), regularity (0.38), abruption of the boarder (0.39) and evaluation of a bright line (0.27). However, evaluation of an intact junction zone was less observer dependent (0.51) with moderate agreement. Page of 40

22 Table 3: Observer variation TVS TVS N Agreement Grey-scale analyses % Kappa (CI 95%) Endometrial texture -Homogenicity 51 74,5 0,49 ( ) -Cystic 51 80,4 0,53 ( ) -If cystic, regular cystic ( ) Endo-myometrial junction -Homogenic ( ) -Regular ( ) -Abrupt ,39 ( ) -Bright line ( ) Diagnosis -Cancer/possible malignant ,66 ( ) -Polyp ( ) - Proliferative/atrophic endometrium ( ) -Hyperplasia ( ) -Main diagnosis benign/malignant ( ) Main diagnosis (all five categories) ( ) Kappa value indicates agreement: <0.20 Poor Fair Moderate Good Very good In four (7.5%) of the patients a GIS could not be performed. In additionally four patients a proper filling of the endometrial cavity was not achieved by the inexperienced observer, allowing 45 (85%) of the patients for final evaluation (Table 4). Excluding the patients whom were labeled not evaluable by both of the observers, left 43 cases for evaluation of homogeneity and echogenicity of the endometrial structure. The observer variation of endometrial texture and endometrial-myometrial junction was at the same level at GIS as at TVS and there was in general no clearer vision of the endometrial-myometrial interface as the numbers of not evaluable examinations were at the same level. There was no larger agreement on a regular or homogeneous endometrial-myometrial junction. However, in the 34 patients where the interface of the endometrial-myometrial junction was clearly evaluable there was a good agreement on abrupt/not abrupt endometrial-myometrial interface (kappa 0.65). There was moderate agreement in terms of deciding whether there was a localized (0.55) or diffuse (0.42) lesion (or both or none) in the endometrial surface and a moderate agreement on whether the endometrial surface was smooth (0.48), polypoid (0.45) or irregular (0.51). In the final evaluation a priority of three diagnoses was allowed When giving priority the observer was asked to plot the most severe diagnosis as the main diagnosis and the lesser severe in secondary and tertiary. In cases where there were doubts about the final diagnosis the observers used the most likely diagnosis as the main diagnosis. Page of 40

23 When a diagnosis of cancer was made in one of the three priorities, the patients were categorized possible malignant. In cases without a diagnosis of malignancy the patients were categorized as not malignant. For a possible malignant diagnosis there was good agreement between the two observers at both TVS (0.66) and GIS (0.73). In the same manner a final diagnosis of benign endometrial polyp, hyperplasia without atypia and proliferative endometrium was made generating new variables. In evaluations with TVS there were moderate agreements between observers of a diagnosis of benign endometrial polyps (0.51), while the agreement rose to good at GIS (0.73). Table 4: Observer variation GIS GIS N Agreement Kappa CI 95% Endometrial texture -Homogeneity 43 76, ( ) -Cystic ( ) Endo-myometrial junction -Homogenic/regular ( ) -Abrupt ( ) -Bright line ( ) Localised lesion ( ) Diffuse lesion ( ) Endometri al surface: -smooth ( ) -polypoid ( ) -irregular ( ) Diagnosis -Cancer/possible malignant ( ) -Polyp ( ) -Hyperplasia ( ) -Proliferative /atrophic endometrium ( ) Main diagnosis benign/malignant ( ) Main diagnosis (all five categories) ( ) % Agreement between the two observers for a main diagnosis of atrophy/proliferative endometrium, benign endometrial polyp, hyperplasia or cancer was fair with TVS (0.38), but good with GIS (0.69). There was no significant difference in the disagreement (McNemars test) between the two examiners with GIS vs. TVS for the main five diagnoses (p=0.07), diagnosis of polyps (p=0.51) or malignant or benign diagnosis (p=0.21). However, for the total number of agreements or disagreements of diagnosis, agreements in the 45 women examined with GIS was statistically higher than in the case with TVS (p= 0.03, Wilcoxon paired signed rank test). The difference between the two observers was (mean (±SD, range)) with TVS: 1.6 mm (±6.4 ( ) and with GIS: 0.7 (± 4.3, ) with regards to measurement of endometrial thickness. Page of 40

24 Scatter plots of endometrial thickness by the two observers with TVS and GIS as well as plots of difference vs. average are seen in Figure 1 and 2 respectively. Histograms and Q-norm plots showed normal distributed measurements. The plot with difference vs. average illustrates that differences could be assumed to be normal distributed without any systematic change by increasing endometrial thickness. There was no significant difference in the measurements of differences (t-test), but there was significant differences in the variance with GIS and with TVS at paired variance analysis (Pitman's variance ratio test) p = Thus there was larger agreement on measurement of endometrial thickness with GIS, than TVS. Figur 5: Bland-Altmann (TVS) Figur 6: Bland-Altmann (GIS) Page of 40