Marije P. van der Paardt, MD Marjolein B. Zagers, MS Regina G. H. Beets-Tan, MD, PhD Jaap Stoker, MD, PhD Shandra Bipat, PhD.

Transcription

1 Note: This copy is for your personal non-commercial use only. To order presentation-ready copies for distribution to your colleagues or clients, contact us at Marije P. van der Paardt, MD Marjolein B. Zagers, MS Regina G. H. Beets-Tan, MD, PhD Jaap Stoker, MD, PhD Shandra Bipat, PhD Patients Who Undergo Preoperative Chemoradiotherapy for Locally Advanced Rectal Cancer Restaged by Using Diagnostic MR Imaging: A Systematic Review and Meta-Analysis 1 Purpose: Materials and Methods: To obtain performance values of magnetic resonance (MR) imaging for restaging locally advanced rectal cancer after neoadjuvant treatment regarding tumor staging, nodal staging, and tumor-free circumferential resection margins (CRMs). MEDLINE, EMBASE, and Cochrane databases were searched for studies regarding restaging compared with a reference standard by using the terms rectal neoplasms, MR imaging, and chemotherapy. The Quality Assessment of Diagnostic Accuracy Studies tool was used, and data on imaging criteria, histopathologic criteria, and restaging were extracted. Responders were defined as positives and nonresponders, as negatives. Mean sensitivity, mean specificity, and positive and negative likelihood ratios (LRs) were determined by using a bivariate random-effects model. A positive LR greater than 5 implied moderate results for responders. Original Research n Evidence-Based Practice 1 From the Department of Radiology, Academic Medical Center, University of Amsterdam, Meibergdreef 9, 1105 AZ Amsterdam, the Netherlands (M.P.v.d.P., M.B.Z., J.S., S.B.); and Department of Radiology, Maastricht University Medical Center, University of Maastricht, Maastricht, the Netherlands (R.G.H.B.T.). Received January 5, 2013; revision requested February 14; revision received March 1; accepted March 20; final version accepted April 1. Address correspondence to M.P.v.d.P. ( m.p.vanderpaardt@ amc.uva.nl). q RSNA, 2013 Results: Conclusion: Thirty-three studies evaluated 1556 patients. For tumor stage, mean sensitivity was 50.4%, mean specificity was 91.2%, positive LR was 5.76, and negative LR was Diffusion-weighted (DW) imaging showed comparable positive LR with significantly improved sensitivity (P =.01) and negative LR (P =.04). Experienced observers showed higher sensitivity (P =.01) and lower negative LR (P =.03) compared with less experienced observers. For CRM, mean sensitivity, mean specificity, positive LR, and negative LR were 76.3%, 85.9%, 5.40, and 0.28, respectively. For nodal stage per patient, mean sensitivity, mean specificity, positive LR, and negative LR were 76.5%, 59.8%, 1.90, and 0.39, respectively; and for nodal stage on a lesion basis, these values were 90.7%, 73.0%, 3.37, and 0.13, respectively. MR imaging showed heterogeneous results of diagnostic performances for restaging rectal cancer after neoadjuvant treatment, but significantly better results were demonstrated when DW imaging was used or with experienced observers. MR imaging can also be used for evaluation of CRM staging, but nodal staging remains challenging. q RSNA, 2013 Supplemental material: /suppl/doi: /radiol /-/dc1 Radiology: Volume 269: Number 1 October 2013 n radiology.rsna.org 101

2 Rectal cancer is a major cause of mortality in the United States, and there were an estimated new cases in 2012 (1,2). Treatment of patients with rectal cancer is based on individual risk factors for recurrence in each patient (3). Patients with a high risk for local recurrence are generally treated with long-term neoadjuvant chemotherapy and radiation therapy (hereafter, chemoradiotherapy) to downstage Advances in Knowledge Studies that evaluated diagnostic performance of MR imaging in restaging of locally advanced rectal cancer after neoadjuvant treatment showed considerable heterogeneity regarding tumor and nodal staging and tumor-free circumferential resection margin (CRM) evaluation. Overall restaging for tumor stage showed a poor mean sensitivity (50.4%) and negative likelihood ratio (LR) (0.54), but a good mean specificity (91.2%) and moderate positive LR (5.76); diffusion-weighted (DW) imaging showed better performance (P =.04) compared with standard MR imaging sequences for restaging of the tumor status of negative results (negative LR, 0.19), without a significant decrease of the positive LR (P =.99). Studies with experienced observers showed significantly better results (higher sensitivity [P =.01] and lower negative LR [P =.03]) compared with studies with less experienced observers for tumor staging. MR imaging showed moderate results for CRM staging (sensitivity, 76.3%; specificity, 85.9%; positive LR, 5.40; negative LR, 0.28). The LRs showed that MR imaging caot discriminate nodal response after chemoradiotherapy (nodal stage per patient: positive LR, 1.90; negative LR, 0.39). and increase the chance of a curative resection (3 5). Accurate restaging is increasingly important for patients with locally advanced rectal cancer undergoing neoadjuvant treatment, because identification of response has major implications for management (6). Magnetic resonance (MR) imaging is a standard technique for local staging of rectal cancer (tumor, lymph node, and circumferential resection margin [CRM] staging) (7) and is also increasingly used for restaging (8). Numerous studies have reported on results of MR imaging for local restaging, but there are considerable differences in methodologic analysis, results, and outcome measures (9 13). Therefore, it is unknown whether MR imaging can be used for restaging. The purpose of our study was to obtain performance values of MR imaging for restaging of locally advanced rectal cancer after neoadjuvant treatment regarding tumor staging, nodal staging, and tumor-free CRM. Materials and Methods A computerized systematic literature search was performed to identify abstracts from studies involving human subjects. The MEDLINE, EMBASE, and Cochrane databases from January 1990 to November 2012 were searched. For MEDLINE and EMBASE, the following keywords were used: rectal neoplasms (medical subject headings search); MR imaging (medical subject headings search); and chemotherapy (text Implications for Patient Care MR imaging for restaging of patients with locally advanced rectal cancer who are undergoing preoperative chemoradiotherapy is difficult to interpret; however, DW imaging and experienced readers seem helpful for improvement of tumor stage evaluation. For evaluation of CRM staging MR imaging can also be used, but MR imaging is still a challenge for nodal staging. word search). For the Cochrane database, we used the following keywords as search terms: rectal cancer (text word search); MR imaging (medical subject headings search); and chemotherapy (text word search). All search hits were independently evaluated by two reviewers (M.B.Z., medical student, and S.B., clinical epidemiologist) and eventually in consensus. M.B.Z. had experience in data extraction from two prospective studies, and S.B. had experience in data extraction from 19 systematic reviews and meta-analyses. All titles and abstracts were screened. Duplicates, reviews, letters, comments, case reports, articles that report other diseases, or other type of results were excluded. The remaining studies were potentially eligible and their full text was retrieved. To identify additional relevant studies, the reference lists of the retrieved studies were checked manually. All potential eligible articles were independently checked for predefined inclusion and exclusion criteria by two reviewers (M.P.v.d.P., research fellow with experience in data extraction in 10 studies, and M.B.Z.). Discrepancies were resolved by consensus. If no consensus could be reached, a third reviewer (S.B.) was consulted. Inclusion criteria were as follows: (a) patients with rectal cancer Published online before print /radiol Content code: Radiology 2013; 269: Abbreviations: CI = confidence interval CRM = circumferential resection margin DW = diffusion weighted LR = likelihood ratio QUADAS = Quality Assessment of Diagnostic Accuracy Studies Author contributions: Guarantors of integrity of entire study, M.P.v.d.P., M.B.Z.; study concepts/study design or data acquisition or data analysis/interpretation, all authors; manuscript drafting or manuscript revision for important intellectual content, all authors; approval of final version of submitted manuscript, all authors; literature research, M.P.v.d.P., M.B.Z., S.B.; statistical analysis, M.P.v.d.P., M.B.Z., S.B.; and manuscript editing, all authors Conflicts of interest are listed at the end of this article. 102 radiology.rsna.org n Radiology: Volume 269: Number 1 October 2013

3 undergoing restaging MR imaging after chemotherapy and/or radiation; (b) restaging for tumor, nodal, and CRM status; (c) results compared with histopathologic findings at rectal surgery, endoscopic findings or follow-up (reference test); and (d) first treatment for rectal cancer. Exclusion criteria were reported data on the same outcome of the same study population (the study with the largest population was included) and 10 or fewer rectal cancer patients. The reviewers (M.P.v.d.P. and M.B.Z.) independently extracted the following data, and discrepancies were resolved in consensus: (a) study design and patient characteristics (ie, year of publication, type of data collection, single or multicenter study, study period, country of origin, department of the first author, consecutive recruitment, number of patients, age, sex ratio, inclusion criteria and exclusion criteria, and staging according to the TNM classification system); (b) the methodologic characteristics of the Quality Assessment of Diagnostic Accuracy Studies (QUADAS) were used (Table 1) because we considered the studies to be diagnostic accuracy studies (14) and the QUADAS tool is used for evaluation of diagnostic accuracy studies, and the advantage of QUADAS is that it considers each methodologic item separately, thus providing the possibility to assess the effect of each separate item on the diagnostic performance, whereas with other tools only the total effect of the combination of the items can be assessed (15); (c) imaging techniques: (ie, magnetic field strength, type of coil used, preparation, use of intravenous contrast agent, imaging sequences, and imaging parameters); (d) image evaluation (ie, number of observers, experience of observers, review time, interobserver agreement, consensus reading, image quality); and (e) reference standard (ie, time between MR imaging and surgery, histopathologic analysis, pathologist experience, composition of the reference standard). Data on restaging were extracted for each imaging sequence and subsequently compared with the reference standard. Different grading systems have been proposed for assessment of rectal Table 1 QUADAS Methodologic Characteristics Used for Study Design Parameter Patient spectrum Selection patients Patient characteristics* Time interval Verification Execution index test Execution reference test standard Evaluation index test Prospective* QUADAS Criteria cancer response to chemoradiotherapy at MR imaging (Table E1 [online]). We extracted response data by using the different grading systems that were reported in the articles. Furthermore, tables were extracted or reconstructed from the available raw data. All data analyses were performed by using software (Microsoft Excel 2000, Microsoft, Redmond, Wash; SPSS 10.0 for Windows, SSPS, Chicago, Ill; SAS 9.2, SAS Institute, Cary, NC). In several studies, multiple datasets were available (eg, multiple readers, multiple sequences). However, because we used all datasets for the data analysis, we adjusted for the correlation by adding the same number for each study in the subject statement of the random-effect approach. This was done for the calculation of the robust standard error, meaning that each study is represented once. P values less than.05 indicated statistically significant difference. Sensitivity and Specificity For each study, we constructed a contingency table for MR imaging to Was the patient spectrum representative of the patients who will receive the test in practice (diagnosed preoperative local advanced rectum tumor)? Were selection criteria clearly described (inclusion and exclusion criteria described in text)? Were patient characteristics described (eg, age, sex)? Is the time period between reference standard and index test short enough to be reasonably sure that the target condition did not change between the two tests (reference standard within 62 days)? Did the whole sample receive verification by using a reference standard of diagnosis (.80%, CRT patients)? Was the execution of the index test described in sufficient detail to permit replication of the test (eg, magnetic field, coils, TR/TE, FOV, section thickness, matrix, b values)? Was the execution of the reference standard described in sufficient detail to permit its replication? Were the index test results interpreted without knowledge of the results of the reference standard? Was the data collected after the research question was defined? Note. Some items from QUADAS were considered irrelevant given the inclusion and exclusion criteria; those items were not assessed. CRT = chemoradiotherapy, FOV = field of view, TE = echo time, TR = repetition time. * This item from the QUADAS tool was not part of the QUADAS tool. compare with the reference standard. Responders were defined per study by following the criteria of Table E1 (online). The relevant diagnostic parameters, sensitivity (true responders divided by all responders), specificity (true nonresponders divided by all nonresponders) were defined. The I 2 statistic, including 95% confidence intervals (CIs), was used for quantification of heterogeneity (16). We used either nonlinear fixed effects (I 2 25%) or random effects (I 2. 25%) approach to obtain summary estimates of sensitivity and specificity (17). Mean logit sensitivity and specificity with corresponding standard errors were obtained, and then antilogit transformation was obtained to calculate summary estimates of sensitivity and specificity with 95% CIs. A summary receiver operating characteristic curve was drawn on the basis of the betweenstudy variance matrix in cases where a negative covariance between the logit sensitivity and logit specificity was obtained from the bivariate random-effect approaches (17). Radiology: Volume 269: Number 1 October 2013 n radiology.rsna.org 103

4 Positive likelihood ratio (LR) and negative LR were calculated with corresponding 95% CIs from the mean logit sensitivity and mean logit specificity, and the corresponding standard errors (17). The following interpretations could be applied to positive LR and negative LR: positive LR greater than 10 and negative LR of less than 0.1 implied large changes; positive LR of 5 10 and negative LR of implied moderate changes; positive LR of 2 5 and negative LR of implied small changes; positive LR greater than 2 and negative LR greater than 0.5 implied tiny changes; and LRs of 1 implied no changes (18). The posttest probabilities for positive results were calculated and plotted against prevalences to study the effect of prevalence. The following factors that can affect diagnostic accuracy and cause heterogeneity were incorporated in the bivariate model: (a) year of publication; (b) representative patient spectrum; (c) clearly described selection criteria; (d) clear description of patient characteristics; (e) short enough time period between reference standard and index test; (f) verification with the reference standard for the whole sample; (g) sufficient description of the index test to permit replication; (h) sufficient description of the reference standard to permit replication; (i) interpretation without knowledge of reference standard; and (j) prospective data collection. Year of publication was explored as continuous factor, and all other factors were explored as binomial (ie, yes, no, or unclear). We considered factors to be explanatory for the observed heterogeneity in diagnostic accuracy if the corresponding regression coefficients were significantly different from zero and P value was less than.05. Subgroup Analysis We evaluated different subgroups according to tumor stage, lymph node stage, and CRM if more than four datasets were available. The following subgroups were defined a priori: diffusion-weighted (DW) imaging group; T0 versus T1 4; T0 2 versus T3 4; time interval less than Figure 1 6 weeks. Because DW imaging was only recently used, articles from 2009 to 2013 were studied. The subgroups on observer experience, consensus versus no consensus, section thickness, malignant nodes greater than 5 mm, and rectal distension were chosen after data analysis. For overall tumor stage, we distinguished the following subgroups: (a) T0 versus T1 4; (b) T0 2 versus T3 4; (c) DW imaging staging; (d) maximum time interval between MR imaging and histologic analysis of 6 weeks; (e) observer with experience of 5 years or more in rectal and/or pelvic MR imaging; (f) consensus versus no consensus; and (g) T2-weighted images with section thickness of 3 mm or less versus greater than 3 mm. For CRM status, we distinguished the following subgroups: studies that used rectal preparation and studies that did not use rectal preparation. For the analysis of nodal status, we divided the data in three subgroups: T2- weighted images with section thickness Figure 1: Flow diagram of articles included in meta-analysis. of 3 mm or less; T2-weighted images with section thickness greater than 3 mm; and upper limit of normal lymph nodes, 5 mm. The z test was performed to analyze differences in logit sensitivity, logit specificity, logit-positive LR, and logit-negative LR estimates between the groups. To study publication bias, we constructed fuel plots for overall tumor and nodal staging on a per-patient basis and CRM status. We placed the natural logarithm of the diagnostic odds ratio on the x-axis and the sample size on the y-axis. The Egger regression test was used to examine fuel plot asymmetry (19). Results We found 264 articles (Fig 1), and of these there were 80 eligible articles. A search resulted in 20 additional eligible articles. Of the 100 eligible studies, 33 (3,9 13,20 46) fulfilled the inclusion criteria (Table E2 [online] for excluded studies). 104 radiology.rsna.org n Radiology: Volume 269: Number 1 October 2013

5 Table 2 Characteristics of Included Studies Study Author, Year of Publication Type of Data Collection, Type of Study Study Period Country of Origin Department of First Author Dalton et al (20), 2012 Retrospective, single center United Kingdom Exeter colorectal unit Song et al (21), 2012 Retrospective, single center Oct 2008 Feb 2009 Republic of Korea Radiology and Center for imaging science Jung et al (3), 2012 Retrospective, single center Nov 2007 July 2009 South Korea Radiology Nougaret et al (9), 2012 Retrospective, single center May 2004 May 2008 France Abdominal imaging Carbone et al (22), 2012 Retrospective, single center NA Italy Radiology Martellucci et al (23), 2012 Prospective, single center Jan 2008 Dec 2009 Italy Surgery Kim et al (24), 2011 Retrospective, single center Aug 2008 Dec 2009 South Korea Radiology Lambregts et al (25), 2011 Prospective, single center Nov 2006 Nov 2009 The Netherlands Radiology Lambregts et al (26), 2011 Retrospective, multicenter The Netherlands Radiology Lambregts et al (27), 2011 Prospective, single center Apr 2008 Dec 2009 The Netherlands Radiology Park et al (28), 2011 Retrospective, single center Oct 2008 Jan 2010 South Korea Radiology Engelen et al (10),2010 Prospective, multicenter Feb 2003 Jan 2008 The Netherlands Surgery Barbaro et al (29), 2009 Prospective, single center May 2004 May 2007 Italy Bioimaging and radiologic sciences Cho et al (30), 2009 NA, NA Apr 2005 Feb 2006 Korea Surgery Dresen et al (31), 2009 Retrospective, multicenter Oct 1999 Aug 2007 The Netherlands Radiology Johnston et al (32), 2009 Retrospective, single center Mar 2003 Jun 2006 United Kingdom Radiology Kim et al (11), 2009 Retrospective, NA Nov 2007 Aug 2008 Korea Radiology Kim et al (33), 2009 Retrospective, single center Jan 2006 Jan 2008 Korea Radiology Lahaye et al (34), 2009 Prospective, multicenter Feb 2003 Jan 2008 The Netherlands Radiology Larsen et al (35), 2009 Prospective, single center Jan 2002 Apr 2007 Norway Surgical oncology Suppiah et al (36), 2009 Retrospective, single center United Kingdom Academic surgical unit Koh et al (37), 2008 Prospective, single center NA United Kingdom Radiology Kulkarni et al (38), 2008 Prospective, multicenter Dec 2001 Dec 2005 United Kingdom Surgery Vliegen et al (39), 2008 Retrospective, multicenter The Netherlands Radiology Allen et al (40), 2007 Retrospective, single center Aug 2000 Sept 2004 United Kingdom Imaging Strassburg et al (41), 2007 Retrospective, single center Nov 2001 Oct 2005 Germany Surgery Baatrup et al (12), 2006 Prospective, NA May 2002 Sept 2004 Norway Surgery Maretto et al (42), 2006 Prospective, NA Jul 2003 Apr 2005 Italy Surgical sciences MERCURY study group (43), 2006 Prospective, multicenter Jan 2002 Oct 2003 United Kingdom NA Chen et al (44), 2005 Prospective, single center Aug 2000 Jun 2003 Taiwan Surgery Denecke et al (45), 2005 Prospective, single center NA Germany Radiation therapy and PET center Blomqvist et al (46), 2002 Retrospective, single center NA Sweden Diagnostic Radiology Hoffman et al (13), 2002 Prospective, NA NA Germany Radiology and radiation oncology Note. NA = not available, PET = positron emission tomography. Data Extraction All studies were performed between 1999 and Fifteen studies were prospective and 17 were retrospective (one study was unknown). From two groups, three articles were included (11,24 27,33) that were specifically scrutinized. We made adjustments in the analyses of two studies of one group (25,27) and two studies of the other group for the overlapping study period (11,33). Nineteen studies were initiated by a department of radiology. Further study characteristics are summarized in Table 2. A total of 1556 patients were included. Median study size was 42 patients (range, ). Mean patient age was 62.1 years (range, 53 71; 28 studies). Male-to-female ratio was 1.9:1 (29 studies). Average radiation dose was 47.5 Gy (range, Gy). In 21 studies, 5-fluorouracil was the most frequently used chemotherapy (Table E3 [online]). Twenty-five (75.8%) studies fulfilled at least seven of the methodologic criteria (Fig 2). Patient spectrum and verification by the reference standard were the most frequently described characteristics, and interpretation of the index test without knowledge of the reference test was the least-described criterion. Data regarding imaging features and image evaluation are outlined in Tables E4 and E5 (online). The majority of studies (n = 27) were performed at 1.5 T. Six studies compared standard T2 sequences with DW imaging. In nine studies (17 datasets), observers had more than 5 years of experience in rectal and/or pelvic MR imaging. Table E6 (online) describes the reference standard and the time between posttreatment MR imaging and reference standard. The interval between posttreatment MR imaging and reference standard ranged from 0 to 181 days. Radiology: Volume 269: Number 1 October 2013 n radiology.rsna.org 105

6 Figure 2 Figure 2: Methodologic criteria of the included articles. Figure 3 shows the sensitivity and specificity of the 58 datasets (in 27 studies; see Table E7 [online]) that evaluated tumor stage. For overall tumor response on posttreatment MR imaging (58 datasets), the respective values of I 2 for sensitivity and specificity were 40.3% (95% CI: 18.9%, 56.0%) and 56.8% (95% CI: 42.5%, 67.6%). The mean sensitivity was 50.4% (95% CI: 38.1%, 62.7%) and mean specificity was 91.2% (95% CI: 85.9%, 94.7%). The corresponding positive LR was 5.76 (95% CI: 3.32, 9.98) and negative LR was 0.54 (95% CI: 0.42, 0.70) (Table 3). Posttest probability results are shown in Figure E1a (online). Figure 4 shows the diagnostic performance of 12 datasets (seven studies; Table E8 [online]) that evaluated CRM status. Histopathologic analysis showed response (ie, tumor-free CRM) in 65.3% (477 of 730) of patients. I 2 value was 86.3% (95% CI: 78.3%, 91.4%) for sensitivity and 46.2% (95% CI: 4.3%, 69.8%) for specificity. Mean sensitivity for CRM response was 76.3% (95% CI: 64.6%, 85.0%) and mean specificity for CRM response was 85.9% (95% CI: 63.2%, 95.6%). Positive LR for CRM response was 5.40 (95% CI: 1.81, 16.09) and negative LR was 0.28 (95% CI: 0.17, 0.44) (Table 3). Posttest probability results are shown in Figure E1b (online). Nodal Staging Figures 5 and E2a (online) demonstrate the diagnostic performance for the 16 datasets (several reviewers and imaging sequences; Table E9 [online]) of studies that evaluated nodal staging per patient. Histologic response was seen in 68.6% (487 of 710) of patients. On a per-patient basis, I 2 was 60.6% (95% CI: 34.5%, 76.3%) for sensitivity and 38.7% (95% CI: 0%, 63.5%) for specificity. Mean sensitivity was 76.5% (95% CI: 67.3%, 83.8%) and specificity was 59.8% (95% CI: 47.2%, 71.3%) (Table 3). Data on response are represented by mean sensitivity and positive LR, whereas data on nonresponse are represented by mean specificity and negative LR. The positive LR on a perpatient basis was 1.90 (95% CI: 1.38, 2.63) and the negative LR was 0.39 (95% CI: 0.26, 0.59) (Table 3). Figures 5 and E2b (online) demonstrate diagnostic performance of 16 datasets (Table E9 [online]) that evaluated nodal staging per lesion. Differentiation between benign and malignant nodes was considered for response. A histologic response was seen in 2694 of 3322 nodes (81.1%). I 2 sensitivity was 89.7% (95% CI: 85.2%, 92.9%) and specificity was 56.8% (95% CI: 27.7%, 74.2%). Mean sensitivity was 90.7% (95% CI: 76.7%, 96.7%) and specificity was 73.0% (95% CI: 67.3%, 78.1%) (Table 3). Nodal staging by differentiation between benign and malignant nodes showed positive LR and negative LR of 3.37 (95% CI: 2.69, 4.22) and 0.13 (95% CI: 0.05, 0.34), respectively (Table 3). Subgroups for Stages The results of the subgroup DW imaging and observer experience are described in more detail; see Table 3 for the summary estimates of the other subgroups. The subgroup DW imaging (five studies, 11 datasets) showed significantly better results for tumor staging compared with standard sequences, mean sensitivity (83.6%; 95% CI: 61.7%, 94.2%; P =.01), and negative LR (0.19; 95% CI: 0.07, 0.51; P =.04). Mean specificity (84.8%; 95% CI: 74.2%, 91.5%; P =.23) and positive LR (5.50; 95% CI: 3.03, 9.95; P =.99) were comparable. Compared with standard MR sequences, DW imaging showed comparable performance after positive results and better performance after negative results. In these studies (composed of 15 datasets), mean sensitivity and mean specificity were 70.0% (95% CI: 53.5%, 82.5%) and 88.2% (95% CI: 80.3%, 93.2%), respectively. Positive LR was 5.93 (95% CI: 3.33, 10.54) and negative LR was 0.34 (95% CI: 0.21, 0.56). Mean sensitivity and negative LR were significantly better compared with studies where the observers had less than 5 years of experience: 47.4% (95% CI: 40.9%, 54.1%; P =.01) and 0.60 (95% CI: 0.53, 0.69; P =.03). Mean specificity and positive LR were comparable: 86.9% (95% CI: 84.1%, 89.2%; P =.72), 3.62 (95% CI: 2.84, 4.60; P =.12). A comparison of subgroup T0 versus T2 4 with subgroup T0 2 versus T3 4 showed that the mean sensitivity, mean specificity, and negative LR of the latter subgroup were significantly better (P,.001, P =.05, and P =.01, 106 radiology.rsna.org n Radiology: Volume 269: Number 1 October 2013

7 Figure 3 Figure 3: Diagnostic performance of studies evaluating tumor stage. (a) Forest plot shows sensitivity and specificity with 95% CI. All studies that evaluated tumor stage are in alphabetical order. Numbers between brackets are 95% CIs. (b) Receiver operating characteristic space shows sensitivity and specificity of individual dataset, which is represented by the boxes. A summary receiver operating characteristic curve was drawn on the basis of the between-study variance matrix obtained from the bivariate random-effect approaches. Conv = conventional sequence, FN = false negative, FP = false positive, R1 = reviewer 1, R2 = reviewer 2, R3 = reviewer 3, TN = true negative, TP = true positive. respectively) (Table 3). In studies performed with a maximal time interval of 6 weeks between MR imaging and histopathologic analysis, mean sensitivity was slightly higher compared with overall tumor staging, but it was not significant. Studies that evaluated results without consensus showed a lower mean sensitivity compared with studies that evaluated results in consensus, but they had a significantly higher mean specificity (P =.01). No significant differences were found for section thickness of 3 mm or smaller compared with section thickness greater than 3 mm (Table 3). No significant differences were found for studies that used rectal distension compared with studies that did not use rectal distension (Table 3). Studies that exclusively considered nodes larger than 5 mm as malignant did not differ significantly from the overall nodal staging per patient. Studies that evaluated nodal stage per patient with a section thickness of 3 mm or smaller did not show statistically significant differences with overall nodal staging per patient. Heterogeneity was only explored for tumor staging because enough datasets were available. Clear description of patient characteristics, a short time period between reference standard and index test, clear description of index test, and interpretation without knowledge of reference standard were significantly associated with diagnostic performance estimates (P,.05). The regression coefficient showed no significant relationship between Radiology: Volume 269: Number 1 October 2013 n radiology.rsna.org 107

8 Table 3 Summary Estimates for Each Restaged Subgroup Response Monitoring Criterion* Mean Sensitivity (%) Mean Specificity (%) Positive LR (%) Negative LR (%) Overall tumor staging (n = 58) 50.4 (38.1, 62.7) 91.2 (85.9, 94.7) 5.76 (3.32, 9.98) 0.54 (0.42, 0.70) Subgroups DW imaging staging (n = 11) 83.6 (61.7, 94.2) 84.8 (74.2, 91.5) 5.50 (3.03, 9.95) 0.19 (0.07, 0.51) Standard sequences of studies published between 2009 and 2013 (n = 49.4 (33.9, 65.0) 91.1 (83.1, 95.5) 5.53 (2.64, 11.59) 0.56 (0.40, 0.77) 31) Observers experience of 5years rectal/pelvic MR imaging (n = 15) 70.0 (53.5, 82.5) 88.2 (80.3, 93.2) 5.93 (3.33, 10.54) 0.34 (0.21, 0.56) II Observers experience of, 5years rectal/pelvic MR imaging (n = 17) 47.4 (40.9, 54.1) 86.9 (84.1, 89.2) 3.62 (2.84, 4.60) 0.60 (0.53, 0.69) T0 vs T1 4 (n = 20) 19.1 (8.3, 38.2) # 94.6 (93.1, 95.9) 3.57 (1.57, 8.11) 0.85 (0.71, 1.03)** T0 2 vs T3 4 (n = 21) 55.3 (40.0, 69.7) 89.8 (82.4, 94.3) 5.44 (2.88, 10.26) 0.50 (0.35, 0.70) Time interval, 6 weeks (n = 31) 64.1 (47.0, 78.2) 91.3 (81.7, 96.1) 7.35 (3.24, 16.69) 0.39 (0.25, 0.62) Consensus (n = 11) 65.6 (43.2, 82.7) 79.4 (74.1, 83.8) 3.18 (2.15, 4.72) 0.43 (0.24, 0.79) No consensus (n = 25) 50.1 (38.2, 61.9) 90.7 (83.8, 94.8) IIII 5.35 (2.89, 9.91) 0.55 (0.43, 0.71) Section thickness 3 mm (n = 22) 47.0 (31.9, 62.5) 89.2 (81.4, 93.9) 4.34 (2.25, 8.36) 0.59 (0.44, 0.81) Section thickness. 3 mm (n = 27) 63.1 (49.8, 74.8) 87.3 (80.1, 92.2) 4.98 (2.98, 8.32) 0.42 (0.30, 0.60) CRM No involvement vs involvement (n = 12) 76.3 (64.6, 85.0) 85.9 (63.2, 95.6) 5.40 (1.81, 16.09) 0.28 (0.17, 0.44) Rectal distension (n = 8) 78.5 (65.9, 87.3) 77.1 (51.8, 91.3) 3.42 (1.41, 8.34) 0.28 (0.16, 0.49) No rectal distension (n = 4) 65.6 (42.8, 82.9) 97.7 (59.8, 99.9) (1.06, 770.0) 0.35 (0.19, 0.65) Nodal staging Nodal staging per patient (n = 16) 76.5 (67.3, 83.8) 59.8 (47.2, 71.3) 1.90 (1.38, 2.63) 0.39 (0.26, 0.59) Nodal staging per lesion, benign vs malignant (n = 16) 90.7 (76.7, 96.7) 73.0 (67.3, 78.1) 3.37 (2.69, 4.22) 0.13 (0.05, 0.34) Malignant nodes. 5 mm (n = 6) 72.0 (64.6, 78.3) 71.1 (61.4, 79.3) 2.49 (1.80, 3.46) 0.39 (0.30, 0.52) Section thickness 3 mm (n = 7) 80.3 (59.0, 92.0) 62.0 (40.5, 79.7) 2.12 (1.18, 3.79) 0.32 (0.13, 0.78) Note. Data in parentheses are 95% CIs, unless otherwise indicated. Response data are represented by mean sensitivity and positive LR, whereas nonresponse data are represented by mean specificity and negative LR. * Data in parentheses are numbers of datasets. Significantly better compared with standard sequences of studies published between 2009 and 2013 (P =.01) Significantly better compared with standard sequences of studies published between 2009 and 2013 (P =.04) Significantly better compared with no experience (P =.01) II Significantly better compared with no experience (P =.03) # Significantly lower compared with overall T staging (P =.01) ** Significantly higher compared with overall T staging (P =.01) Significantly better compared with T0 versus T1 4 (P,.001) Significantly better compared with T0 versus T1 4 (P =.05) Significantly better compared with T0 versus T1 4 (P =.01) IIII Significantly better compared with consensus (P =.01) sample size and natural logarithm of the diagnostic odds ratio. The coefficients for overall tumor staging, CRM, nodal staging per patient, and nodal staging per lesion were 1.8 (95% CI: 25.3, 1.8), 1.4 (95% CI: 24.7, 7.6), 2.3 (95% CI: 26.6, 11.2), and 22.2 (95% CI: 234.9, 30.4), respectively. Discussion This systematic review and metaanalysis shows that restaging with MR imaging of rectal cancer after preoperative chemoradiotherapy is challenging. Overall restaging for tumor stage showed a poor mean sensitivity (50.4%) and negative LR (0.54), but it also showed a good mean specificity (91.2%) and positive LR (5.76). It is difficult to differentiate fibrosis from residual tumor (24). For overall tumor staging, positive MR imaging results were more likely to correctly predict which patients had responded to therapy than were negative MR imaging results likely to predict which patients had not responded to therapy. Restaging with DW imaging demonstrated a good mean sensitivity (83.6%), and an improvement of the negative LR, without a decrease of the positive LR and specificity (84.8% and 5.50, respectively). Evaluation of a tumor-free CRM showed a mean sensitivity of 76.3% and mean specificity of 85.9% with a corresponding positive LR of 5.40 and a negative LR of 0.28, with moderate to good posttest probabilities. 108 radiology.rsna.org n Radiology: Volume 269: Number 1 October 2013

9 Although the results of mean sensitivities for nodal staging (76.5% per-patient nodal staging and 91.7% for benign vs malignant nodes) were better compared with the results for tumor staging, mean specificities for nodal staging were moderate (59.8%, 73.0%). The LRs show that MR imaging caot discriminate nodal response after chemoradiotherapy treatment. These results show that MR imaging can be used for evaluation of the tumor status and involvement of the CRM after neoadjuvant treatment; however, MR imaging is not reliable for evaluation of nodal involvement. It is known that nodal staging in rectal cancer is challenging due to high prevalence of malignancy in small nodes (47). Section thickness of 3 mm or less and a stringent cutoff value for malignant nodes (.5 mm) did not seem to improve the diagnostic performance. We aimed to minimize some of the well-known limitations of meta-analysis. To reduce the risk of missing important studies, we searched MEDLINE as well as additional databases (EMBASE, Cochrane) with broad search terms and manually checked the reference lists of included articles. Independent review and extraction of data were performed by two reviewers. Studies that reported duplicate data were excluded unless the reported data were on different status (tumor, CRM, and lymph node stage). Furthermore, stringent inclusion criteria were used. The major limitation of this review was the extent of observed heterogeneity. We used a random effect approach to analyze the heterogeneous data. Nevertheless, the heterogeneity in this type of diagnostic study remains a concern, and to some extent it influences the certainty of the conclusions. Four methodologic factors (clear description of patient characteristics, short enough time period between reference standard and index test, clear description of index test, and interpretation without knowledge of reference standard per QUADAS) were significantly associated with performance estimates of tumor, lymph nodes, and CRM status. Heterogeneity is also caused by complex Figure 4 Figure 4: Diagnostic performance of studies that evaluated CRM status. (a) Forest plot shows sensitivity and specificity with 95% CI. Studies that evaluated CRM are in alphabetical order. Numbers between brackets are 95% CIs. (b) Receiver operating characteristic space shows sensitivity and specificity of individual dataset, which is represented by the boxes. A negative covariance between the logit sensitivity and logit specificity was not obtained; therefore, no summary receiver operating characteristic curve could be drawn. Conv = conventional sequence, FN = false negative, FP = false positive, R1 = reviewer 1, R2 = reviewer 2, R3 = reviewer 3, TN = true negative, TP = true positive. differences in study characteristics, such as different definitions of a malignant lymph node and response and the use of different chemoradiotherapy treatment regimens. Technical developments and the use of different imaging techniques might influence restaging. We therefore studied the effect of DW imaging because it is increasingly used and enough data were available (7). A significantly higher diagnostic performance was found for more experienced observers. The time interval between the posttreatment MR imaging and the reference test varied considerably (0 181 days), and changes in disease status can occur in this time frame (48). Because of these known changes, we described a subgroup with a maximum 6-week Radiology: Volume 269: Number 1 October 2013 n radiology.rsna.org 109

10 Figure 5 preoperative chemoradiotherapy in this systematic review and meta-analysis. MR imaging showed moderate results for tumor staging, with significantly better results when DW imaging was used or with experienced observers and also moderate results were seen for restaging of CRM, but nodal staging remains a challenge. Disclosures of Conflicts of Interest: M.P.v.d.P. Financial activities related to the present article: money paid to author from United European Gastroenterology Week 2012 for travel grant. Financial activities not related to the present article: none to disclose. Other relationships: none to disclose. M.B.Z. No relevant conflicts of interest to disclose. R.G.B.T. No relevant conflicts of interest to disclose. J.S. Financial activities related to the present article: none to disclose. Financial activities not related to the present article: money paid to author s institution from Robarts Clinical Trials for consultancy. Other relationships: none to disclose. S.B. No relevant conflicts of interest to disclose. References 1. Siegel R, DeSantis C, Virgo K, et al. Cancer treatment and survivorship statistics, CA Cancer J Clin 2012;62(4): Figure 5: Diagnostic performance of studies that evaluated lymph node stage. (a) Forest plot shows sensitivity and specificity with 95% CI of lymph nodes stage per patient. All studies that evaluated lymph nodes stage are in alphabetical order. Numbers between brackets are 95% CIs. (b) Forest plot shows lymph node stage per lesion. ADC = apparent diffusion coefficient, conv = conventional sequence, FN = false negative, FP = false positive, R1 = reviewer 1, R2 = reviewer 2, R3 = reviewer 3, TN = true negative, TP = true positive. 2. The Surveillance, Epidemiology, and End Results (SEER) Cancer Statistics Review , National Cancer Institute. Published April Accessed December Jung SH, Heo SH, Kim JW, et al. Predicting response to neoadjuvant chemoradiation therapy in locally advanced rectal cancer: diffusion-weighted 3 Tesla MR imaging. J Magn Reson Imaging 2012;35(1): interval between MR imaging and histopathologic analysis. Better results were seen if compared with the results of the overall tumor staging, but they were not statistically significant. We also evaluated the following two subgroups with different tumor staging definitions of response: T0 versus T1 4 and T0 2 versus T3 4. The subgroups showed different results that were statistically significant. Recent studies have reported on a wait-and-see policy in a selection of patients with evidence of a complete response (T0 on imaging and endoscopy) after neoadjuvant chemoradiotherapy (48,49). High diagnostic performance for discriminating T0 would therefore be essential; however, 8. Lahaye MJ, Engelen SM, Nelemans PJ, et al. Imaging for predicting the risk factors the circumferential resection margin and nodal disease of local recurrence in rectal canat present a wait-and-see policy is not standard practice. Therefore, the differentiation between T0 2 versus T3 4 seems to be the most useful application for clinical practice. From the literature (50), it is known that the use of intrarectal material can overestimate CRM involvement. Although a higher mean specificity and positive LR were found in studies that evaluated the CRM without intrarectal material, these differences were not significant. Few studies reported on excluded examinations due to poor image quality; this may have biased the results. In conclusion, we reviewed the role of MR imaging in restaging of patients with locally advanced rectal cancer undergoing 4. Cammà C, Giunta M, Fiorica F, Pagliaro L, Craxì A, Cottone M. Preoperative radiotherapy for resectable rectal cancer: A meta-analysis. JAMA 2000;284(8): Elferink MA, van Steenbergen LN, Krijnen P, et al. Marked improvements in survival of patients with rectal cancer in the Netherlands following changes in therapy, Eur J Cancer 2010;46(8): Evans J, Patel U, Brown G. Rectal cancer: primary staging and assessment after chemoradiotherapy. Semin Radiat Oncol 2011; 21(3): Beets-Tan RG, Beets GL. Rectal cancer: review with emphasis on MR imaging. Radiology 2004;232(2): radiology.rsna.org n Radiology: Volume 269: Number 1 October 2013

11 cer: a meta-analysis. Semin Ultrasound CT MR 2005;26(4): Nougaret S, Rouanet P, Molinari N, et al. MR volumetric measurement of low rectal cancer helps predict tumor response and outcome after combined chemotherapy and radiation therapy. Radiology 2012; 263(2): Engelen SM, Beets-Tan RG, Lahaye MJ, et al. MRI after chemoradiotherapy of rectal cancer: a useful tool to select patients for local excision. Dis Colon Rectum 2010; 53(7): Kim SH, Lee JM, Hong SH, et al. Locally advanced rectal cancer: added value of diffusion-weighted MR imaging in the evaluation of tumor response to neoadjuvant chemo- and radiation therapy. Radiology 2009;253(1): Baatrup G, Pfeiffer P, Svolgaard B, Jensen HA. Resectability of rectal cancers still fixed after radio-chemotherapy: evaluation by digital rectal examination, MRI, and intraoperative examination. Int J Colorectal Dis 2006; 21(1): Hoffma KT, Rau B, Wust P, et al. Restaging of locally advanced carcinoma of the rectum with MR imaging after preoperative radiochemotherapy plus regional hyperthermia. Strahlenther Onkol 2002;178(7): Whiting P, Rutjes AW, Reitsma JB, Bossuyt PM, Kleijnen J. The development of QUA- DAS: a tool for the quality assessment of studies of diagnostic accuracy included in systematic reviews. BMC Med Res Methodol 2003;3: Whiting P, Harbord R, Kleijnen J. No role for quality scores in systematic reviews of diagnostic accuracy studies. BMC Med Res Methodol 2005;5: Higgins JP, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency in metaanalyses. BMJ 2003;327(7414): Reitsma JB, Glas AS, Rutjes AW, Scholten RJ, Bossuyt PM, Zwinderman AH. Bivariate analysis of sensitivity and specificity produces informative summary measures in diagnostic reviews. J Clin Epidemiol 2005;58(10): Evidence-Based Diagnosis, Office of Medical Education Research and Development College of Human Medicine Michigan State University. Diagnosis/Diagnosis6.html. Published Accessed May Song F, Khan KS, Dies J, Sutton AJ. Asymmetric fuel plots and publication bias in meta-analyses of diagnostic accuracy. Int J Epidemiol 2002;31(1): Dalton RS, Velineni R, Osborne ME, et al. A single-centre experience of chemoradiotherapy for rectal cancer: is there potential for nonoperative management? Colorectal Dis 2012;14(5): Song I, Kim SH, Lee SJ, Choi JY, Kim MJ, Rhim H. Value of diffusion-weighted imaging in the detection of viable tumour after neoadjuvant chemoradiation therapy in patients with locally advanced rectal cancer: comparison with T2 weighted and PET/CT imaging. Br J Radiol 2012;85(1013): Carbone SF, Pirtoli L, Ricci V, et al. Assessment of response to chemoradiation therapy in rectal cancer using MR volumetry based on diffusion-weighted data sets: a preliminary report. Radiol Med (Torino) 2012;117(7): Martellucci J, Scheiterle M, Lorenzi B, et al. Accuracy of transrectal ultrasound after preoperative radiochemotherapy compared to computed tomography and magnetic resonance in locally advanced rectal cancer. Int J Colorectal Dis 2012;27(7): Kim SH, Lee JY, Lee JM, Han JK, Choi BI. Apparent diffusion coefficient for evaluating tumour response to neoadjuvant chemoradiation therapy for locally advanced rectal cancer. Eur Radiol 2011;21(5): Lambregts DM, Beets GL, Maas M, et al. Accuracy of gadofosveset-enhanced MRI for nodal staging and restaging in rectal cancer. A Surg 2011;253(3): Lambregts DM, Vandecaveye V, Barbaro B, et al. Diffusion-weighted MRI for selection of complete responders after chemoradiation for locally advanced rectal cancer: a multicenter study. A Surg Oncol 2011; 18(8): Lambregts DM, Maas M, Riedl RG, et al. Value of ADC measurements for nodal staging after chemoradiation in locally advanced rectal cancer-a per lesion validation study. Eur Radiol 2011;21(2): Park MJ, Kim SH, Lee SJ, Jang KM, Rhim H. Locally advanced rectal cancer: added value of diffusion-weighted MR imaging for predicting tumor clearance of the mesorectal fascia after neoadjuvant chemotherapy and radiation therapy. Radiology 2011;260(3): Barbaro B, Fiorucci C, Tebala C, et al. Locally advanced rectal cancer: MR imaging in prediction of response after preoperative chemotherapy and radiation therapy. Radiology 2009;250(3): Cho YB, Chun HK, Kim MJ, et al. Accuracy of MRI and 18F-FDG PET/CT for restaging after preoperative concurrent chemoradiotherapy for rectal cancer. World J Surg 2009; 33(12): Dresen RC, Beets GL, Rutten HJ, et al. Locally advanced rectal cancer: MR imaging for restaging after neoadjuvant radiation therapy with concomitant chemotherapy. Part I. Are we able to predict tumor confined to the rectal wall? Radiology 2009;252(1): Johnston DF, Lawrence KM, Sizer BF, et al. Locally advanced rectal cancer: histopathological correlation and predictive accuracy of serial MRI after neoadjuvant chemotherapy. Br J Radiol 2009;82(976): Kim SH, Lee JM, Park HS, Eun HW, Han JK, Choi BI. Accuracy of MRI for predicting the circumferential resection margin, mesorectal fascia invasion, and tumor response to neoadjuvant chemoradiotherapy for locally advanced rectal cancer. J Magn Reson Imaging 2009;29(5): Lahaye MJ, Beets GL, Engelen SM, et al. Locally advanced rectal cancer: MR imaging for restaging after neoadjuvant radiation therapy with concomitant chemotherapy. Part II. What are the criteria to predict involved lymph nodes? Radiology 2009; 252(1): Larsen SG, Wiig JN, Emblemsvaag HL, et al. Extended total mesorectal excision in locally advanced rectal cancer (T4a) and the clinical role of MRI-evaluated neo-adjuvant downstaging. Colorectal Dis 2009;11(7): Suppiah A, Hunter IA, Cowley J, et al. Magnetic resonance imaging accuracy in assessing tumour down-staging following chemoradiation in rectal cancer. Colorectal Dis 2009;11(3): Koh DM, Chau I, Tait D, Wotherspoon A, Cuingham D, Brown G. Evaluating mesorectal lymph nodes in rectal cancer before and after neoadjuvant chemoradiation using thin-section T2-weighted magnetic resonance imaging. Int J Radiat Oncol Biol Phys 2008;71(2): Kulkarni T, Gollins S, Maw A, Hobson P, Byrne R, Widdowson D. Magnetic resonance imaging in rectal cancer downstaged using neoadjuvant chemoradiation: accuracy of prediction of tumour stage and circumferential resection margin status. Colorectal Dis 2008;10(5): Vliegen RF, Beets GL, Lammering G, et al. Mesorectal fascia invasion after neoadjuvant chemotherapy and radiation therapy for locally advanced rectal cancer: accuracy of MR imaging for prediction. Radiology 2008;246(2): Radiology: Volume 269: Number 1 October 2013 n radiology.rsna.org 111

12 40. Allen SD, Padhani AR, Dzik-Jurasz AS, Glye-Jones R. Rectal carcinoma: MRI with histologic correlation before and after chemoradiation therapy. AJR Am J Roentgenol 2007;188(2): Strassburg J, Lewin A, Ludwig K, et al. Optimised surgery (so-called TME surgery) and high-resolution MRI in the plaing of treatment of rectal carcinoma. Langenbecks Arch Surg 2007;392(2): Maretto I, Pomerri F, Pucciarelli S, et al. The potential of restaging in the prediction of pathologic response after preoperative chemoradiotherapy for rectal cancer. A Surg Oncol 2007;14(2): MERCURY Study Group. Diagnostic accuracy of preoperative magnetic resonance imaging in predicting curative resection of rectal cancer: prospective observational study. BMJ 2006;333(7572): Chen CC, Lee RC, Lin JK, Wang LW, Yang SH. How accurate is magnetic resonance imaging in restaging rectal cancer in patients receiving preoperative combined chemoradiotherapy? Dis Colon Rectum 2005;48(4): Denecke T, Rau B, Hoffma KT, et al. Comparison of CT, MRI and FDG-PET in response prediction of patients with locally advanced rectal cancer after multimodal preoperative therapy: is there a benefit in using functional imaging? Eur Radiol 2005;15(8): Blomqvist L, Holm T, Nyrén S, Svanström R, Ulvskog Y, Iselius L. MR imaging and computed tomography in patients with rectal tumours clinically judged as locally advanced. Clin Radiol 2002;57(3): Wang C, Zhou Z, Wang Z, et al. Patterns of neoplastic foci and lymph node micrometastasis within the mesorectum. Langenbecks Arch Surg 2005;390(4): Habr-Gama A, Perez RO, Nadalin W, et al. Operative versus nonoperative treatment for stage 0 distal rectal cancer following chemoradiation therapy: long-term results. A Surg 2004;240(4): ; discussion Maas M, Beets-Tan RG, Lambregts DM, et al. Wait-and-see policy for clinical complete responders after chemoradiation for rectal cancer. J Clin Oncol 2011;29(35): Slater A, Halligan S, Taylor SA, Marshall M. Distance between the rectal wall and mesorectal fascia measured by MRI: Effect of rectal distension and implications for preoperative prediction of a tumour-free circumferential resection margin. Clin Radiol 2006;61(1): radiology.rsna.org n Radiology: Volume 269: Number 1 October 2013