Pulmonary Nodule Size Evaluation with Chest Tomosynthesis 1

Transcription

1 Note: This copy is for your personal non-commercial use only. To order presentation-ready copies for distribution to your colleagues or clients, contact us at Åse A. Johnsson, MD, PhD Erika Fagman, MD Jenny Vikgren, MD, PhD Valeria A. Fisichella, MD, PhD Marianne Boijsen, MD, PhD Agneta Flinck, MD, PhD Susanne Kheddache, MD, PhD Angelica Svalkvist, PhD Magnus Båth, PhD Pulmonary Nodule Size Evaluation with Chest Tomosynthesis 1 Purpose: Materials and Methods: To evaluate intra- and interobserver variability, as well as agreement for nodule size measurements on chest tomosynthesis and computed tomographic (CT) images. The Regional Ethical Review Board approved this study, and all participants gave written informed consent. Thirty-six segmented nodules in 20 patients were included in the study. Eight observers measured the left-to-right, inferior-to-superior, and longest nodule diameters on chest tomosynthesis and CT images. Intra- and interobserver repeatability, as well as agreement between measurements on chest tomosynthesis and CT images, were assessed as recommended by Bland and Altman. Original Research n Thoracic Imaging 1 From the Departments of Radiology (A.A.J., E.F., J.V., V.A.F., M. Boijsen, A.F., S.K.) and Radiation Physics (A.S., M. Båth), the Sahlgrenska Academy at University of Gothenburg, Bruna Straket 11, SE Gothenburg, Sweden; and Departments of Radiology (A.A.J., E.F., J.V., V.A.F., M. Boijsen, A.F., S.K.) and Medical Physics and Biomedical Engineering (A.S., M. Båth), Sahlgrenska University Hospital, Gothenburg, Sweden. Received July 20, 2011; revision requested August 25; revision received January 25, 2012; accepted February 24; final version accepted May 8. Supported by the Swedish Research Council [2011/488], Swedish Radiation Safety Authority [2008/2232, 2009/1689, 2010/4363, 2012/2021], King Gustav V Jubilee Clinic Cancer Research Foundation [2007:28, 2008:50], Swedish Federal Government under the LUA/ALF agreement [ALFGBG ], and Health & Medical Care Committee of the Region Västra Götaland [VGFOUREG-12046, VGFOUREG-27551, VGFOUREG-81341]. Address correspondence to A.A.J. ( ase.johnsson@vgregion.se). q RSNA, 2012 Results: Conclusion: The difference between the mean manual and the segmented diameter was 22.2 and 22.3 mm for left-to-right and 22.6 and 22.2 mm for the inferior-to-superior diameter for measurements on chest tomosynthesis and CT images, respectively. Intraobserver 95% limits of agreement (LOA) for the longest diameter ranged from a lower limit of 21.1 mm and an upper limit of 1.0 mm to 21.8 and 1.8 mm for chest tomosynthesis and from 20.6 and 0.9 mm to 23.1 and 2.2 mm for axial CT. Interobserver 95% LOA ranged from 21.3 and 1.5 mm to 22.0 and 2.1 mm for chest tomosynthesis and from 21.8 and 1.1 mm to 22.2 and 3.1 mm for axial CT. The 95% LOA concerning the mean of the observers measurements of the longest diameter at chest tomosynthesis and axial CT were 62.1 mm (mean measurement error, 0 mm). For the different observers, the 95% LOA between the modalities ranged from 22.2 and 1.6 mm to 23.2 and 2.8 mm. Measurements on chest tomosynthesis and CT images are comparable, because there is no evident bias between the modalities and the repeatability is similar. The LOA between measurements for the two modalities raise concern if measurements from chest tomosynthesis and CT were to be used interchangeably. q RSNA, 2012 Supplemental material: /suppl/doi: /radiol /-/dc1 Radiology: Volume 265: Number 1 October 2012 n radiology.rsna.org 273

2 Size estimates of pulmonary tumors, metastases, and nodules play an important role in thoracic imaging. For example, the Response Evaluation Criteria in Solid Tumors (1) which uses unidimensional manual measurements of the longest tumor diameter is considered the method of choice for the morphologic assessment of tumor response to treatment. Furthermore, with regard to incidentally detected nodules, size is the main criterion for the decision on follow-up according to Fleischner Society guidelines (2). Chest tomosynthesis is a recently available imaging modality that provides some of the tomographic benefits of computed tomography (CT) at a substantially reduced radiation dose and cost compared with CT (3,4). A study from Duke University reported that 74% of nodules greater than 4 mm depicted on CT images (section thickness, 1.25 mm) were visible on chest tomosynthesis images (3). In another study, Vikgren et al (4) found that 92% of all nodules greater than 4 mm and all nodules greater than 6 mm depicted on thick-section (5 mm) CT images were perceptible on chest tomosynthesis images. Because the majority of nodules considered actionable according to Fleischner Society guidelines (2) are detectable on chest tomosynthesis images, chest tomosynthesis has been suggested as an alternative method for follow-up of pulmonary nodules (5). However, so far, no data have been available for the measurement accuracy Advances in Knowledge nn Repeatability of manual measurements on chest tomosynthesis and CT images is comparable for the two modalities, with limits of agreement (LOA) for intraobserver variability of approximately 61.5 mm averaged over all nodules. nn The LOA between measurements for the two modalities are wider than the LOA for intraobserver variability, in the order of 62.5 mm, for an average observer. of clinical nodules on chest tomosynthesis images in the scientific literature; although a phantom study of nodule size evaluation with chest tomosynthesis suggested that nodule size measurements on chest tomosynthesis images might be an alternative to measurements on CT images (6). Nevertheless, in some parts of the world where chest tomosynthesis is used in clinical practice, selected patients with pulmonary nodules already are being followed up by using chest tomosynthesis, replacing some or all CT examinations for that particular patient. Consequently, the present study was designed to investigate measurements on chest tomosynthesis images with regard to repeatability and agreement in comparison to measurements on CT images. Comparable repeatability in terms of intra- and interobserver variability for measurements from the two imaging modalities would support the use of either modality for follow-up examination. If the agreement between measurements at chest tomosynthesis and CT is clinically acceptable, the modalities might even be used interchangeably. The aims of the present study were to evaluate intra- and interobserver variability, as well as agreement for nodule size measurements on chest tomosynthesis and CT images. Materials and Methods The Regional Ethical Review Board approved this study, and all participants gave written informed consent. Patients with known malignancies scheduled for chest CT were invited to the study, Implication for Patient Care nn The relevant for interchangeable use of measurements from chest tomosynthesis and CT for follow-up of pulmonary nodules calls for caution; although there seems to be no systematic bias between the modalities, the LOA between chest tomosynthesis and CT are wider than the LOA for intraobserver variability for each modality. where chest tomosynthesis examination was performed in connection to the chest CT. Patients Twenty consecutive patients with pulmonary metastases or nodules graded as clearly visible on chest tomosynthesis images by two experienced thoracic radiologists (J.V. and A.A.J., 18 and 12 years of experience in thoracic radiology, respectively) were included in the study. Eighteen patients had clinically diagnosed pulmonary metastases. The 20 patients represented 15 different types of cancers (testicular [n = 3], breast [n = 2], pancreatic [n = 2], adenoid cystic carcinoma [n = 2], rectal [n = 1], malignant melanoma [n = 1], etc). Two patients had Hodgkin disease with probably unrelated pulmonary nodules. There were 11 women with a mean age of 49 years (age range, years) and nine men with a mean age of 52 years (age range, years). All examinations were performed during a 6-month period. CT Examination Chest CT examinations were performed with 16- or 64-dectector equipment (LightSpeed Pro 16 and LightSpeed VCT; GE Healthcare, Milwaukee, Wis). Scan parameters were 140 kv and ma for both imagers. For the 16 detector row scanner, the detector Published online before print /radiol Content code: Radiology 2012; 265: Abbreviations: LOA = limits of agreement SD = standard deviation Author contributions: Guarantors of integrity of entire study, A.A.J., S.K., A.S.; study concepts/study design or data acquisition or data analysis/interpretation, all authors; manuscript drafting or manuscript revision for important intellectual content, all authors; manuscript final version approval, all authors; literature research, A.A.J., J.V., S.K., A.S.; clinical studies, A.A.J., J.V., V.A.F., M. Boijsen, A.F., S.K., A.S.; statistical analysis, A.A.J., S.K., A.S., M. Båth; and manuscript editing, all authors Relevant conflicts of interest are listed at the end of this article. 274 radiology.rsna.org n Radiology: Volume 265: Number 1 October 2012

3 configuration was 16 detectors mm section gap, and pitch was 1.375:1. For the 64 detector row scanner, the detector configuration was 64 detectors mm section gap, and pitch was 0.984:1. Images (1.25-mm section thickness) were reconstructed with an interval of 0.6 mm. The effective dose for a standard patient (170 cm and 70 kg) was 4 msv (4). Chest Tomosynthesis Examination Chest tomosynthesis examinations were performed with VolumeRAD (GE Healthcare, Chalfont St Giles, England) within an average of 27 minutes (range, 4 85 minutes) of the chest CT. For a chest tomosynthesis acquisition with this system, the detector position is fixed, whereas the x-ray tube performs a vertical continuous movement around the standard orthogonal posteroanterior projection. Within 10 seconds, 60 low-dose projections from 215 to 15 were collected at a tube voltage of 120 kv. The low-dose projections were used to reconstruct coronal section images of the chest every 5 mm, resulting in approximately 60 section images for a normal-sized patient. The average effective dose for a chest tomosynthesis examination in a standard patient was 0.13 msv (7). Nodule Selection and Study Images CT images were used for automated nodule measurements with a commercially available program (Lung Volume Computed Assisted Reading, or Lung VCAR; GE Healthcare, Milwaukee, Wis) and multiplanar reformations at a workstation (Advantage, windows 4.4; GE Healthcare, Milwaukee, Wis). In each patient, two experienced thoracic radiologists (J.V. and A.A.J.), in consensus, selected nodules that were judged as adequately segmented by Lung VCAR (ie, not including adjacent structures such as vessels or pleura) and identified the corresponding nodules on the chest tomosynthesis images. An illustration of a segmented nodule is given in Figure 1a. After successful segmentation of a nodule, the CT images (section thickness, 1.25 mm) were used Figure 1 Figure 1: (a) CT image shows nodule judged as adequately segmented. (b d) Corresponding axial CT, coronal CT, and chest tomosynthesis images of the same nodule. to create multiplanar reformation image stacks in coronal and axial planes containing the nodule. The coronal reconstructions had a section thickness of 1.2 mm with a 0.2-mm overlap and a display field of view of 20 cm. Axial multiplanar reformation images were reconstructed without overlap with a section thickness of 0.7 mm for examinations performed with the 16 detector row scanner and 0.6 mm for examinations performed with the 64 detector row scanner. The display field of view was cm, depending on patient size. In total, 36 solid nodules that were judged as adequately segmented and categorized as well defined were included in the study, with a mean of 1.8 nodules per patient (range, one to four nodules). An additional 20 nodules were judged as not adequately segmented, because adjacent structures such as vessels or pleura were included in the segmented volume, and therefore were excluded. For each nodule, three separate image stacks containing the nodule were created one coronal CT, one axial CT, and one coronal chest tomosynthesis. Examples are given in Figure 1b 1d. Measurement Study To compare nodule size estimates on CT and chest tomosynthesis images, an observer study was conducted. Eight observers five radiologists with experience ranging from 6 to 33 years and three medical students with Radiology: Volume 265: Number 1 October 2012 n radiology.rsna.org 275

4 some training in radiology and manual diameter measurements participated in the study. A detailed description of the experience of the radiologists is given in Table E1 (online). The 108 image stacks of the 36 nodules were presented to the observers by using ViewDEX ( (8,9). ViewDEX is software specially developed for conducting observer performance studies, which uses a random number generator to present a given set of images in a unique random order to each observer. The measurements were performed on a Digital Imaging and Communications in Medicine calibrated high-resolution medical display (RadiForce G33; Eizo Nanao, Ishikawa, Japan). The instruction to the observers was to measure the left-to-right diameter and the longest diameter of the nodules in all image stacks, as well as the inferiorto-superior diameter on chest tomosynthesis and coronal CT images. The measurements were not confined to a specific image of the nodule. The window width and window level were 1400 and 2400 HU for CT and and 8192 for chest tomosynthesis. These window settings are used in the clinical routine at our institution and could not be changed by the observers. The observers were free to use the magnification tool. For the evaluation of intraobserver variability, measurements were repeated in a new individual random order after a period of at least 3 weeks. The randomization was performed with the ViewDEX software (8,9). Statistical Analysis The manual measurement data of the left-to-right and inferior-to-superior diameter from chest tomosynthesis were plotted against both automated and manual measurement data from CT, and the line of equality was drawn. Measurement errors for manual measurements on chest tomosynthesis and CT images compared with automated measurement were calculated for each observer. Results were presented as means with the standard deviation (SD) and the 95% confidence interval of the mean. The method described by Bland and Altman (10) for assessing agreement Table 1 Descriptive Statistics of the Assessment of Nodule Diameters by Using Automated and Manual Measurements Nodule Diameter (n = 36) Segmented Measurement (mm) between two methods of clinical measurement was used to investigate intra- and interobserver repeatability, as well as the agreement between measurements on chest tomosynthesis and CT images, for manual measurements of the clinically used longest diameter. The difference between measurements was plotted against the mean of the measurements. The mean difference and the SD of the differences were calculated, as well as the 95% limits of agreement (LOA), by using the mean difference 6 2 SD. The precision of estimated LOA depends on sample size. The standard error of the 95% LOA is (3SD 2 /n), where SD is the SD of the differences and n is the sample size (10). The 95% lower and upper LOA are therefore presented with their 95% confidence intervals. The corrected SD of measurement differences was applied for repeated measurements (10) when comparing the means of the observers measurements on chest tomosynthesis and CT images. With regard to interobserver variability, the SD across observers was determined for each nodule diameter and modality. The Wilcoxon signed rank test was then applied to the sets of SDs to test if there was a statistically significant difference between the modalities ( faculty.vassar.edu/lowry/wilcoxon.html). On the basis of the obtained results, a retrospective power analysis was performed ( Manual CT Measurement (mm)* Left-to-right Mean Minimum Maximum Inferior-to-superior Mean Minimum Maximum * Data are means of the eight observers manual measurements on CT and chest tomosynthesis images. html). A P value less than.05 was considered to indicate a significant difference. Results Manual Chest Tomosynthesis Measurement (mm)* To reduce recall bias, all reported manual nodule diameter measurements, except for the intraobserver agreement, refer to the first measurement of the observers. The segmented left-to-right diameter of the nodules ranged from 4.2 to 28 mm (mean, 10.7 mm). The segmented inferior-to-superior diameter ranged from 3.9 to 30 mm (mean, 10.4 mm). Descriptive statistics of the measurements obtained by using manual measurements on chest tomosynthesis and CT images in comparison to the automated measurements on CT images are in Table 1. Manual measurements on both chest tomosynthesis and CT images underestimated nodule size compared with automated volumetry. Figure 2 shows the actual manual measurement data from chest tomosynthesis images of the left-to-right and inferior-to-superior diameters for all nodules and observers plotted against automated and manual measurement data from CT images. The agreement between manual measurements at chest tomosynthesis and automated measurements at CT was inferior to the agreement between manual measurements at chest tomosynthesis and CT performed by the individual observer. The mean measurement errors for the different 276 radiology.rsna.org n Radiology: Volume 265: Number 1 October 2012

5 Figure 2 Figure 2: Plots show (a) manual measurements of the left-to-right (L-R) diameter of the nodules on chest tomosynthesis (CTS) images by the observers against CT segmented measurements, (b) manual measurements of inferior-to-superior (I-S) diameter of the nodules on chest tomosynthesis images by the observers against CT segmented measurements, (c) manual measurements of left-to-right diameter of the nodules on chest tomosynthesis images by the observers against corresponding manual measurements by each observer on axial CT images, and (d) manual measurements of inferior-to-superior diameter of the nodules on chest tomosynthesis images by the observers against corresponding manual measurements by each observer on coronal CT images. In all plots, the 45 line of equality is drawn to help assess agreement between the measurements. Plots illustrate that the agreement between manual measurements at chest tomosynthesis and automated measurements at CT is inferior to the agreement between manual measurements at chest tomosynthesis and CT performed by the individual observer. observers, by using the segmented leftto-right diameter as reference, ranged from 21.9 to 22.4 mm for chest tomosynthesis, from 21.4 to 22.4 mm for coronal CT, and from 21.9 to 22.6 mm for axial CT. For the inferior-tosuperior diameter, the mean measurement errors ranged from 22.4 to 22.9 mm for chest tomosynthesis and from 21.9 to 22.6 mm for coronal CT. The mean measurement errors for each observer and every diameter are in Table E2 (online). The intraobserver 95% LOA for measurements of the longest diameter, by using the mean of the two measurements as reference, ranged from 21.1 to 1.0 mm for the least variable observer to 61.8 mm for the most variable observer for measurements at chest tomosynthesis. For measurements at axial CT, the intraobserver 95% LOA ranged from 20.6 to 0.9 mm to 23.1 to 2.2 mm for the least and most variable observer, respectively. The intraobserver 95% LOA for all observers and each image type for measurements of the longest diameter are in Table 2. The interobserver 95% LOA concerning the estimates of the longest nodule diameter for each possible pair of radiologists ranged from 21.3 to 1.5 mm for the least variable pair of radiologists to 22.0 to 2.1 mm for the most variable pair of radiologists for measurements at chest tomosynthesis. For measurements on axial CT, the interobserver 95% LOA ranged from 21.8 to 1.1 mm to 22.2 to 3.1 mm for the least and most variable pair of observers, respectively. The interobserver 95% LOA for all observers and each image type are in Table 3. Assessing the agreement between measurements of the longest diameter at chest tomosynthesis and CT by the difference between the observers mean measurement of nodule diameter for the two modalities rendered mean measurement errors of 20.2 mm (95% confidence interval: 20.4, 0.1) with 95% LOA of 21.7 to 1.4 mm for comparison of chest tomosynthesis to coronal CT and 0.0 mm (95% confidence interval: 20.3, 0.3) with 95% LOA of 22.1 to 2.1 mm for comparison of chest tomosynthesis to axial CT. Bland- Altman plots illustrate the agreement in Figure 3. The 95% LOA for measurements of the longest diameter on chest tomosynthesis and coronal CT images for the individual observers ranged from 21.4 to 1.9 mm for the least variable observer to 22.6 to 2.3 mm for the most variable observer. The corresponding 95% LOA for measurements on chest tomosynthesis and axial CT images were 22.2 to 1.6 mm and 23.2 to 2.8 mm, respectively. The 95% LOA for all observers regarding measurements of the longest diameter at chest tomosynthesis and CT are in Table 4. Bland-Altman plots of the agreement between measurements on chest tomosynthesis and coronal CT images for the most and least experienced radiologist as well as their intraobserver variability for each of the two imaging modalities are in Figure 4. With regard to the interobserver variability, the mean of the SD across Radiology: Volume 265: Number 1 October 2012 n radiology.rsna.org 277

6 Table 2 Intraobserver Error and 95% LOA for All Observers and Each Image Type for Measurements of Longest Diameter Chest Tomosynthesis Coronal CT Axial CT Observer TR (20.3, 0.3) 21.8 (22.3, 21.3) 1.8 (1.3, 2.3) 0.0 (20.3, 0.3) 21.8 (22.3, 21.3) 1.8 (1.3, 2.3) 0.0 (20.3, 0.3) 21.7 (22.2, 21.2) 1.7 (1.2, 2.2) TR (20.1, 0.3) 21.2 (21.5, 20.8) 1.4 (1.1, 1.8) 0.0 (20.3, 0.3) 21.7 (22.1, 21.2) 1.7 (1.2, 2.2) 0.1 (20.1, 0.3) 21.0 (21.3, 20.7) 1.2 (0.9, 1.5) TR (20.2, 0.3) 21.3 (21.7, 20.9) 1.4 (1.0, 1.8) 20.1 (20.4, 0.3) 22.1 (22.7, 21.6) 2.0 (1.4, 2.6) 0.1 (20.2, 0.3) 21.3 (21.7, 20.9) 1.5 (1.1, 1.9) GR (20.2, 0.1) 21.1 (21.4, 20.8) 1.0 (0.7, 1.3) 20.1 (20.3, 0.1) 21.3 (21.7, 21.0) 1.1 (0.8, 1.5) 20.4 (20.8, 0.0) 23.1 (23.8, 22.3) 2.2 (1.5, 3.0) GR (0.0, 0.4) 21.2 (21.6, 20.8) 1.6 (1.2, 1.9) 0.2 (0.0, 0.3) 20.6 (20.8, 20.4) 1.0 (0.7, 1.2) 0.2 (0.0, 0.3) 20.6 (20.8, 20.4) 0.9 (0.7, 1.1) MS (20.2, 0.2) 21.3 (21.7, 21.0) 1.3 (0.9, 1.6) 0.3 (0.1, 0.6) 21.1 (21.5, 20.7) 1.8 (1.4, 2.2) 0.0 (20.2, 0.3) 21.5 (22.0, 21.1) 1.5 (1.1, 2.0) MS (20.3, 0.3) 21.6 (22.0, 21.1) 1.6 (1.1, 2.0) 0.1 (20.1, 0.4) 21.4 (21.9, 21.0) 1.6 (1.2, 2.1) 0.0 (20.2, 0.3) 21.5 (21.9, 21.0) 1.5 (1.1, 1.9) MS (20.7, 0.2) 21.8 (22.2, 21.4) 0.9 (0.5, 1.3) 20.3 (20.5, 20.1) 21.7 (22.0, 21.3) 1.0 (0.7, 1.4) 20.4 (20.6, 20.2) 21.5 (21.8, 21.2) 0.7 (0.4, 1.0) Note. The mean of the two measurements of every nodule on each image type by the individual observer was used as reference. Data in parentheses are 95% confidence intervals. GR = general radiologist, MS = medical student, TR = thoracic radiologist. Table 3 Interobserver Error and 95% LOA for All Possible Pairs of Radiologists and Each Image Type for Measurements of Longest Diameter Chest Tomosynthesis Coronal CT Axial CT Observer TR 1 and TR (20.3, 0.4) 22.0 (22.6, 21.5) 2.1 (1.5, 2.7) 20.3 (20.5, 0.0) 21.9 (22.4, 21.4) 1.3 (0.9, 1.8) 20.2 (20.5, 0.2) 22.3 (22.9, 21.7) 1.9 (1.3, 2.5) TR 1 and TR (20.4, 0.2) 22.0 (22.5, 21.5) 1.7 (1.5, 2.3) 20.5 (20.8, 20.2) 22.3 (22.8, 21.8) 1.3 (0.8, 1.8) 20.3 (20.6, 0.0) 22.1 (22.7, 21.6) 1.6 (1.1, 2.1) TR 1 and GR (20.8, 20.2) 22.5 (23.0, 21.9) 1.5 (0.9, 2.0) 0.3 (0.0, 0.5) 21.2 (21.6, 20.8) 1.7 (1.3, 2.1) 0.5 (0.0, 0.9) 22.2 (23.0, 21.5) 3.1 (2.4, 3.9) TR 1 and GR (20.3, 0.2) 21.8 (22.3, 21.3) 1.7 (1.2, 2.2) 0.3 (0.0, 0.5) 21.1 (21.5, 20.7) 1.7 (1.3, 2.0) 0.1 (0.3, 0.5) 22.3 (23.0, 21.6) 2.6 (1.9, 3.2) TR 2 and TR (20.1, 0.4) 21.4 (21.8, 20.9) 1.7 (1.3, 2.2) 20.2 (20.6, 0.1) 22.6 (23.2, 21.9) 2.1 (1.4, 2.7) 20.1 (20.3, 0.2) 21.6 (22.0, 21.2) 1.4 (1.0, 1.9) TR 2 and GR (20.8, 20.3) 22.0 (22.4, 21.6) 0.9 (0.5, 1.3) 0.5 (0.2, 0.8) 21.5 (22.0, 20.9) 2.5 (2.0, 3.1) 0.7 (0.4, 0.9) 21.1 (22.6, 20.6) 2.4 (1.9, 2.9) TR 2 and GR (20.4, 0.2) 21.8 (22.2, 21.3) 1.6 (1.1, 2.1) 0.5 (0.2, 1.0) 21.3 (21.8, 20.8) 2.4 (1.9, 2.9) 0.3 (0.0, 0.6) 21.4 (21.9, 20.9) 2.0 (1.5, 2.5) TR 3 and GR (20.6, 0.1) 21.9 (22.3, 21.4) 1.1 (0.7, 1.6) 0.8 (0.6, 1.0) 20.5 (20.8, 20.1) 2.0 (1.7, 2.4) 0.8 (0.4, 1.1) 21.3 (21.9, 20.8) 2.9 (2.3, 3.4) TR 3 and GR (20.1,0.3) 21.3 (21.7, 20.9) 1.5 (1.1, 1.8) 20.8 (21.1, 20.5) 22.8 (23.3, 22.2) 1.2 (0.7, 1.8) 20.4 (20.7, 20.1) 22.2 (22.8, 21.7) 1.4 (0.9, 2.0) GR 1 and GR (20.7, 20.2) 22.1 (22.5, 21.6) 1.2 (0.7, 1.6) 0.0 (20.2, 0.2) 21.3 (21.7, 20.9) 1.3 (0.9, 1.7) 20.4 (20.6, 20.1) 21.8 (22.2, 21.4) 1.1 (20.7, 1.5) Note. The mean of the measurements by the two radiologists of every nodule on each image type was used as reference. Data in parentheses are 95% confidence intervals. Data are from the first measurement by each radiologist. GR = general radiologist, TR = thoracic radiologist. 278 radiology.rsna.org n Radiology: Volume 265: Number 1 October 2012

7 Table 4 Error and 95% LOA for Measurements of Longest Diameter at Chest Tomosynthesis in Comparison to Coronal and Axial CT Images for All Observers Observer Chest Tomosynthesis Compared with Coronal CT Chest Tomosynthesis Compared with Axial CT TR (20.3, 0.4) 22.2 (22.8, 21.5) 2.3 (1.6, 2.9) 0.2 (20.2, 0.6) 22.3 (23.1, 21.6) 2.7 (2.0, 3.4) TR (20.6, 0.2) 22.6 (23.3, 22.0) 2.3 (1.6, 2.9) 0.0 (20.4, 0.4) 22.5 (23.3, 21.8) 2.6 (1.9, 3.3) TR (20.9, 20.2) 22.6 (23.2, 22.0) 1.4 (0.9, 2.0) 20.2 (20.7, 0.3) 23.2 (24.0, 22.4) 2.8 (2.0, 3.6) GR (20.5, 0.1) 22.2 (22.8, 21.7) 1.8 (1.3, 2.4) 20.1 (20.3, 0.6) 22.5 (23.2, 21.7) 2.7 (2.0, 3.5) GR (0.0, 0.5) 21.4 (21.9, 20.9) 1.9 (1.5, 2.4) 0.2 (20.1, 0.6) 22.1 (22.8, 21.4) 2.6 (1.9, 3.2) MS (20.2, 0.4) 21.8 (22.3, 21.3) 1.9 (1.4, 2.5) 20.1 (20.5, 0.4) 22.6 (23.3, 21.9) 2.5 (1.8, 3.2) MS (20.9, 20.1) 22.8 (23.5, 22.2) 1.9 (1.2, 2.5) 20.3 (20.6, 0.0) 22.2 (22.7, 21.7) 1.6 (1.0, 2.1) MS (20.4, 0.2) 21.9 (22.4, 21.4) 1.6 (1.1, 2.1) 0.1 (20.3, 0.4) 22.1 (22.7, 21.5) 2.2 (1.6, 2.8) Note. The mean of the measurements at chest tomosynthesis and CT of every nodule by each observer was used as reference. Data in parentheses are 95% confidence intervals. Data are from the first measurement. GR = general radiologist, MS = medical student, TR = thoracic radiologist. Figure 3 Figure 3: (a, b) Bland-Altman plots show the difference between the means of the observers manual measurements of the longest nodule diameter on chest tomosynthesis (CTS) and CT images against the average of the means of the same diameter. Plots show agreement between measurements of the longest nodule diameter on chest tomosynthesis and (a) coronal and (b) axial CT images. Dashed line (center) = mean of differences (20.2 [95% confidence interval: 20.4, 0.1] and 0.0 [95% confidence interval: 20.3, 0.3] in a and b, respectively). Dotted lines (top and bottom) = upper and lower limits of agreement (mean difference, 62 SD). Corrected SD of measurement differences was used because of repeated measurements of the same nodule (10). observers at chest tomosynthesis ranged from 0.53 mm (left-to-right diameter) to 0.56 mm (longest diameter). At CT, the mean of the SD across observers ranged from 0.43 mm (inferiorto-superior diameter) to 0.59 mm (longest diameter at axial CT). The largest observed difference between modalities in SD across observers was 0.11 mm (inferior-to-superior diameter; chest tomosynthesis: 0.54 mm, CT: 0.43 mm). There were no statistically significant differences between the modalities for any diameter measurement according to the Wilcoxon signed rank test (P >.1 for all comparisons). The retrospective power analysis revealed that the sample sizes required to achieve statistical significance for the observed difference between the modalities, regarding the SD across observers, ranged from 67 Radiology: Volume 265: Number 1 October 2012 n radiology.rsna.org 279

8 Figure 4 Figure 4: (a, b) Bland-Altman plots show the difference between the manual measurements of the longest nodule diameter on chest tomosynthesis and coronal CT images against the average of the measurements for the (a) most and (b) least experienced radiologist. (c, d) Bland-Altman plots of intraobserver variability for the (c) most and (d) least experienced radiologist measurements of the longest nodule diameter at chest tomosynthesis. (e, f) Bland-Altman plots of intraobserver variability for the (e) most and (f) least experienced radiologist measurements of the longest nodule diameter at coronal CT. (c f) Plots show the difference between the first and second manual measurements of the longest nodule diameter against the average of the measurements. Dashed line (center) = mean of differences. Dotted lines (top and bottom) = upper and lower limits of agreement (mean difference, 62 SD). Plots show that LOA between measurements at chest tomosynthesis and CT for both observers are wider than the corresponding intraobserver variability for the two modalities. 280 radiology.rsna.org n Radiology: Volume 265: Number 1 October 2012

9 nodules (inferior-to-superior diameter) to 3458 nodules (longest diameter). The P values, observed differences, power, minimal detectable differences, and sample size required for each diameter are in Table E3 (online). Discussion Size estimates play an important role in radiology (1,2), and both systematic measurement errors and variability of measurements are equally important parameters. The new modality chest tomosynthesis has been suggested as an alternative method for follow-up of pulmonary nodules (5). The present study was designed to investigate measurements at chest tomosynthesis regarding geometric dimensions, repeatability, and agreement in comparison to measurements at CT. One major drawback in measurement studies on clinical nodules is that the true sizes of the studied objects are unknown. We therefore decided to use the left-to-right and inferiorto-superior diameter derived from the segmentation process for comparison to have a standardized situation compared with means of the observers measurements. However, for the clinically used longest nodule diameter, the means of the observers measurements still had to be used for comparison, because Lung VCAR does not report this parameter. The use of manual diameter measurements as size estimates of pulmonary nodules and metastases at CT has been questioned, and automated volumetry has been recommended (11 13). However, for automated lung nodule volume, the measurement precision and accuracy depend on a number of factors (14), and at present there is no option of lung nodule volumetry with chest tomosynthesis. Another problem is that all nodules cannot be correctly segmented. Ashraf et al (15) reported that nodule segmentation was successful in 72% of cases. Only 64% of the nodules in the present study were judged as adequately segmented. Manual measurements on both CT and chest tomosynthesis images underestimated the left-to-right and inferiorto-superior diameter compared with the segmentation process. The mean difference for the left-to-right diameter was 22.3 mm on axial CT and 22.2 mm on chest tomosynthesis images. For the inferior-to-superior diameter, the mean difference was 22.2 mm on coronal CT and 22.6 mm on chest tomosynthesis images. Although great care was taken to exclude nodules where the segmentation process obviously included parts of vessels or pleura, small parts of adjacent structures may still have been present, resulting in overestimation of the segmented diameters. From a phantom study, it has been shown that nodule size overestimations with the VCAR program do occur, especially with small nodules (16). In a phantom study of measurements at chest tomosynthesis, a limited underestimation (mean relative measurement error, 21.1%) of the true diameter was found, whereas CT resulted in overestimation of nodule size (mean relative measurement error, 1.4%) (6). With regard to clinical nodules, both underestimation (17) and overestimation (18) of manual nodule size estimates in comparison to automated volumetry have been reported. Concerning the, to some extent, larger underestimation of the manual inferior-to-superior diameter measurements on chest tomosynthesis images in the present study, one could speculate that in-plane artifacts that can occur in the scan direction of the chest tomosynthesis system affected the delineation of the nodule. These artifacts have been studied in breast tomosynthesis and depend on both size and contrast of the object (19). Although one would expect that this artifact would make the nodule appear larger, the deblurring algorithm used for chest tomosynthesis in this study produces a halo-like phenomenon in the craniocaudal direction, which was not included in the measurement of the nodule by the observers. With regard to repeatability of the measurements, the interobserver 95% LOA for unidimensional measurements of the longest nodule diameter on axial CT images for the least variable pair of radiologists was 21.8 to 1.1 mm, which is comparable to 21.5 to 1.9 mm for two observers reported by Marten et al (13) and 21.7 to 1.7 mm reported by Revel et al (11). Intraobserver agreement was of the same magnitude. In a phantom study (6), measurement variability expressed as SD of the measurement error was reported to be 0.5 mm for measurements on axial CT images and 0.3 mm for measurements on chest tomosynthesis images. In the present study, the SD of the measurement error for the different observers ranged from 1.9 to 2.6 mm for measurements on axial CT images and from 2.2 to 2.6 mm for measurements on chest tomosynthesis images when using the segmented left-to-right diameter as reference. Thus, the results in this study do not confirm the finding of less variability for measurements at chest tomosynthesis as indicated in the phantom study. A possible reason for this could be the partly irregular appearance of clinical nodules or metastases compared with spheres in a phantom, although all nodules included in the present study were categorized as well defined. For interobserver variability, the retrospective power analysis revealed that differences between modalities in SD across observers in the order of 0.2 mm would have been detected as statistically significant, given the sample size used in the present study (Table E3 [online]). To put this result into a clinical perspective, concerning nodules 10 mm or larger, at least a 20% increase in nodule size is required to establish progressive disease according to the Response Evaluation Criteria in Solid Tumors (1). The opinion of the authors is therefore that a clinically relevant difference in interobserver variability, had it existed, would have been revealed in the present study. Because nodule growth occurs in any direction, an important limitation of chest tomosynthesis is that size estimates only can be performed in one plane (coronal or sagittal), whereas CT provides the possibility for diameter measurements in all planes as well as for automated volumetry. Nevertheless, it is important to establish whether manual measurements on chest tomosynthesis images can be regarded as acceptable size estimates. When assessing the agreement between measurements of the longest diameter on chest tomosynthesis and axial CT images, of which the latter has been the basis of pulmonary nodule size estimates for Radiology: Volume 265: Number 1 October 2012 n radiology.rsna.org 281

10 decades, the confidence interval of the mean measurement error was 20.3 to 0.3 (ie, no systematic error was found). When assessing the agreement between measurements of the longest diameter on chest tomosynthesis and coronal CT images, which in theory should correspond more accurately to the chest tomosynthesis measurements, a slight underestimation was observed (mean measurement error, 20.2 mm), although this was not statistically significant as the confidence interval of the mean measurement error was 20.4 to 0.1. One could speculate that the fact that the coronal CT images were presented to the readers in a larger magnification than the chest tomosynthesis images could have affected the results. Although the readers were free to use the magnification tool, all readers did not choose to use this tool. Manual measurements are variable (11 13), and consistency can be improved if the same reader performs serial measurements of a nodule or tumor (20). In the present study, the LOA for every observer regarding measurements on chest tomosynthesis and CT images were wider than their corresponding intraobserver variability for the modalities, indicating that the use of modalities interchangeably calls for caution. There were several limitations of the present study. The selection of nodules (ie, nodules that were not clearly visible or judged as not adequately segmented were excluded) resulted in inclusion of a limited number of well-defined solid nodules. The lack of part-solid and ground-glass nodules might have affected the results. Furthermore, the study did not include nodule detection, and some nodules may not have been detected if only chest tomosynthesis had been used. However, in a clinical situation, follow-up with chest tomosynthesis would not be suggested unless the nodule was clearly observable. Another important limitation of the study was that the key parameter regarding follow-up of pulmonary nodules (ie, change in size) was not evaluated. In conclusion, the results of the present study indicated that manual measurements on chest tomosynthesis and CT images are comparable, from a clinical view point, because there is no evident bias between the modalities and the repeatability is similar. However, the LOA between measurements from the two modalities raise concern if measurements from chest tomosynthesis and CT were to be used interchangeably for follow-up of pulmonary nodules. Acknowledgments: The authors thank the medical students Anders Båth, Christian Johansson, and Jacob Zeuchner for their participation in the study. Disclosures of Relevant Conflicts of Interest: A.A.J. No relevant conflicts of interest to disclose. E.F. No relevant conflicts of interest to disclose. J.V. Financial activities related to the present article: none to disclose. Financial activities not related to the present article: author receives payment for lectures from GE (GE symposium ECR 2011). Other relationships: none to disclose. V.A.F. No relevant conflicts of interest to disclose. M. Boijsen. No relevant conflicts of interest to disclose. A.F. No relevant conflicts of interest to disclose. S.K. No relevant conflicts of interest to disclose. A.S. No relevant conflicts of interest to disclose. M. Båth. No relevant conflicts of interest to disclose. References 1. Eisenhauer EA, Therasse P, Bogaerts J, et al. New response evaluation criteria in solid tumours: revised RECIST guideline (version 1.1). Eur J Cancer 2009;45(2): MacMahon H, Austin JH, Gamsu G, et al. Guidelines for management of small pulmonary nodules detected on CT scans: a statement from the Fleischner Society. Radiology 2005;237(2): Dobbins JT 3rd, McAdams HP, Song JW, et al. Digital tomosynthesis of the chest for lung nodule detection: interim sensitivity results from an ongoing NIH-sponsored trial. Med Phys 2008;35(6): Vikgren J, Zachrisson S, Svalkvist A, et al. Comparison of chest tomosynthesis and chest radiography for detection of pulmonary nodules: human observer study of clinical cases. Radiology 2008;249(3): Dobbins JT 3rd, McAdams HP. Chest tomosynthesis: technical principles and clinical update. Eur J Radiol 2009;72(2): Johnsson ÅA, Svalkvist A, Vikgren J, et al. A phantom study of nodule size evaluation with chest tomosynthesis and computed tomography. Radiat Prot Dosimetry 2010;139(1-3): Båth M, Svalkvist A, von Wrangel A, Rismyhr-Olsson H, Cederblad Å. Effective dose to patients from chest examinations with tomosynthesis. Radiat Prot Dosimetry 2010;139(1-3): Börjesson S, Håkansson M, Båth M, et al. A software tool for increased efficiency in observer performance studies in radiology. Radiat Prot Dosimetry 2005;114(1-3): Håkansson M, Svensson S, Zachrisson S, Svalkvist A, Båth M, Månsson LG. View- DEX: an efficient and easy-to-use software for observer performance studies. Radiat Prot Dosimetry 2010;139(1-3): Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986;1(8476): Revel MP, Bissery A, Bienvenu M, Aycard L, Lefort C, Frija G. Are two-dimensional CT measurements of small noncalcified pulmonary nodules reliable? Radiology 2004;231(2): Bogot NR, Kazerooni EA, Kelly AM, Quint LE, Desjardins B, Nan B. Interobserver and intraobserver variability in the assessment of pulmonary nodule size on CT using film and computer display methods. Acad Radiol 2005;12(8): Marten K, Auer F, Schmidt S, Kohl G, Rummeny EJ, Engelke C. Inadequacy of manual measurements compared to automated CT volumetry in assessment of treatment response of pulmonary metastases using RECIST criteria. Eur Radiol 2006;16(4): Gavrielides MA, Kinnard LM, Myers KJ, Petrick N. Noncalcified lung nodules: volumetric assessment with thoracic CT. Radiology 2009;251(1): Ashraf H, de Hoop B, Shaker SB, et al. Lung nodule volumetry: segmentation algorithms within the same software package cannot be used interchangeably. Eur Radiol 2010;20(8): Ravenel JG, Leue WM, Nietert PJ, Miller JV, Taylor KK, Silvestri GA. Pulmonary nodule volume: effects of reconstruction parameters on automated measurements a phantom study. Radiology 2008;247(2): Pauls S, Kürschner C, Dharaiya E, et al. Comparison of manual and automated size measurements of lung metastases on MDCT images: relevant influence on therapeutic decisions. Eur J Radiol 2008;66(1): Christe A, Torrente JC, Lin M, et al. CT screening and follow-up of lung nodules: effects of tube current-time setting and nodule size and density on detectability and of tube current-time setting on apparent size. AJR Am J Roentgenol 2011;197(3): Svahn T, Ruschin M, Hemdal B, et al. Inplane artifacts in breast tomosynthesis quantified with a novel contrast-detail phantom. Proc SPIE 2007;6510:65104R. 20. Erasmus JJ, Gladish GW, Broemeling L, et al. Interobserver and intraobserver variability in measurement of non-small-cell carcinoma lung lesions: implications for assessment of tumor response. J Clin Oncol 2003;21(13): radiology.rsna.org n Radiology: Volume 265: Number 1 October 2012