CME. Original Research

Transcription

1 CME JOURNAL OF MAGNETIC RESONANCE IMAGING 38: (2013) Original Research Interstudy Variability in Cardiac Magnetic Resonance Imaging Measurements of Ventricular Volume, Mass, and Ejection Fraction in Repaired Tetralogy of Fallot: A Prospective Observational Study Shannon E. Blalock, MD, 1,2 Puja Banka, MD, 1,2 Tal Geva, MD, 1,2 Andrew J. Powell, MD, 1,2 Jing Zhou, MS, 1 and Ashwin Prakash, MD 1,2 * Purpose: To assess the interstudy variability of cardiac magnetic resonance imaging (CMR) parameters of ventricular size and function in repaired tetralogy of Fallot (TOF). Materials and Methods: Patients with TOF (n ¼ 30, median age 23.5 years, 43% male) were enrolled prospectively. Each patient underwent two consecutive CMR examinations on the same day. Each examination was analyzed for ventricular size and function by two observers and multiple comparisons were made with assessment of agreement using Bland Altman analysis and intraclass correlation coefficients (ICC). Results: Agreement for most measures of ventricular size and function was high when a single observer analyzed both studies. Agreement was worse when different observers analyzed sequential studies. This effect was most prominent on measurements of right ventricular (RV) mass and there was slight improvement when mass was measured during systole. Aside from ventricular mass, agreement was similar for RV and left ventricular (LV) parameters. Conclusion: CMR measures of ventricular size and function have acceptable repeatability across serial examinations in patients with repaired TOF. Measurements of RV mass are subject to higher variability. For most parameters, agreement limits are wider when measurements are performed by multiple operators. These results will aid in the interpretation of study-to-study variations in the follow-up of individual patients and in designing future clinical trials. 1 Department of Cardiology, Boston Children s Hospital, Boston, Massachusetts, USA. 2 Department of Pediatrics, Harvard Medical School, Boston, Massachusetts, USA. *Address reprint requests to: A.P., Department of Cardiology, Boston Children s Hospital, 300 Longwood Ave., Boston, MA ashwin.prakash@cardio.chboston.org Received November 5, 2012; Accepted December 19, DOI /jmri View this article online at wileyonlinelibrary.com. Key Words: tetralogy of Fallot; magnetic resonance imaging; ventricular function; variability J. Magn. Reson. Imaging 2013;38: VC 2013 Wiley Periodicals, Inc. IN PATIENTS WITH repaired tetralogy of Fallot (TOF), right (RV) and left ventricular (LV) dilation and dysfunction are associated with adverse clinical outcomes including heart failure, ventricular tachycardia, and sudden cardiac death (1 5). Cardiac magnetic resonance imaging (CMR) is commonly used for serial surveillance of ventricular size and function and to inform management decisions such as the timing of pulmonary valve replacement (6 9). Therefore, knowledge of the reproducibility of these measurements is important to distinguish between technique- and operator-related differences versus disease-related changes. The intra- and interobserver variability in quantifying RV size and function from the same image dataset has been previously reported (6,7). However, during serial follow-up of patients, additional measurement variability, interstudy variability, may be introduced by differences in scan planes, spatial resolution, slice thickness, and number of slices. The magnitude and significance of this interstudy variability and its interaction with interobserver differences have not been previously described. The aim of this study, therefore, was to prospectively assess the interstudy variability in CMR measurements of ventricular size, function, and mass in patients with repaired TOF and to study the interactions between interstudy variability and interobserver differences. MATERIALS AND METHODS Subjects Patients with repaired TOF scheduled for a clinically indicated CMR at a single institution between March 2009 and April 2010 were enrolled prospectively after VC 2013 Wiley Periodicals, Inc. 829

2 830 Blalock et al. written informed consent. Patients scheduled for CMR under sedation and/or anesthesia were not eligible for participation. Those thought to have significant imaging artifact related to ferromagnetic implants or arrhythmia were excluded. The study protocol was approved by departmental and institutional ethics and scientific review committees. Clinical Data Demographic data and patient s weight, height, and body surface area (BSA) were recorded at enrollment. Additional clinical data were abstracted from the medical records. The underlying anatomic diagnoses and type of and age at each surgical procedure were noted. CMR Technique Each patient underwent two consecutive CMR examinations performed by different, blinded technologists with a 5 10-minute interval between studies. Briefly, after the clinical examination was completed the first technologist removed the patient from the scanner and left the scanning area. A second technologist blinded to the first examination then repositioned the patient into the scanner and acquired localizing images, and ventricular short- and long-axis images. All imaging was performed using a 1.5T whole body scanner (Achieva, Philips Medical Systems, Best, The Netherlands). Electrocardiogram (ECG)-gated steadystate free precession imaging sequences (echo time msec, repetition time msec, flip angle 45, turbo factor 10 20, reconstructed images per cardiac cycle 30) and standardized protocol for function assessment were used (8,9). Briefly, 12 contiguous short-axis slices perpendicular to the ventricular long-axis (slice thickness 6 8 mm, interslice distance 0 2 mm) were obtained during brief (10 20 seconds) periods of breath holding. Additional slices were acquired as needed to provide complete coverage of very large ventricles. CMR data were reviewed using a commercially available computer workstation (Extended Workstation, Philips Medical Systems). Long- and short-axis cine images of the ventricles were cross-referenced to facilitate separation of atria from ventricles on basal slices (6). RV and LV end-diastolic (EDV) and end-systolic volumes (ESV), masses, stroke volumes (SV), and ejection fractions (EF) were measured on a personal computer using commercially available software (MASS, Medis, Leiden, The Netherlands) as previously described (8,10). Each examination was analyzed by two observers with multiple comparisons. Figure 1 summarizes the three scenarios of interobserver and interstudy comparisons analyzed in this study. For comparisons B and C, contours from Examination 1 were available to the investigator analyzing Examination 2 (matched comparison). For measurements of ventricular mass, an additional set of analyses were performed to evaluate whether interobserver variability was lower when the Figure 1. Summary of study comparisons. Comparison A: Same image dataset analyzed by two observers without access to the other s contours (interobserver variability). Comparison B: Two datasets analyzed by the same observer who was allowed to compare contours between studies (interstudy variability). Comparison C: Two datasets analyzed by two observers (one of whom was the clinical reader) who were allowed to compare contours between studies (interstudy and interobserver variability). measurement was made on an end-systolic frame rather than an end-diastolic frame. For this analysis, epicardial and endocardial contours were drawn on end-systolic frames by two blinded observers on Examination 2 for each subject. Statistical Analyses Agreement between measurements was assessed using techniques described by Bland and Altman (11). Agreement for each measurement was expressed as 2 standard deviations (SD) of the interstudy differences (repeatability coefficient). This reflects the maximum expected within-patient difference between two measurements and a lower value is desirable. Interstudy variability was also assessed using intraclass correlation coefficients (ICC) estimated with variance components models. The ICC indicates the proportion of variability explained by subject differences as opposed to interstudy variability, observer differences, or random error. ICC values for each RV and LV measurement were compared using methods described by Donner (12). Statistical analyses were performed using SAS software v. 9.2 (Cary, NC). RESULTS Subjects Demographic and clinical characteristics of the 30 study subjects are summarized in Table 1. A majority of patients were young adults who had their TOF repaired an average of years before CMR. A transannular patch was used for surgical repair in a majority of patients. CMR measurements of

3 Variability in CMR measurements in TOF 831 Table 1 Patient Characteristics (n¼30) Variable Value Age at CMR (years) 23.5 (11 64) Male gender (%) 13 (43%) Weight (kg) 68.9 (33 113) Height (cm) 166 ( ) Body surface area (m 2 ) 1.85 ( ) Type of surgical repair Transannular patch 16 (53%) RV-PA conduit 5 (17%) Valve-sparing repair 6 (20%) Pulmonary valve replacement 3 (10%) Data are median (range) or number (percent). RV-PA, right ventricle-to-pulmonary artery. ventricular size and function are summarized in Table 2. In general, ventricular size and function parameters were typical for repaired TOF with normal LV parameters and dilated RV with borderline or mild systolic dysfunction. Interstudy Variability Measurements made by a single observer on Examinations 1 and 2 were compared for each patient (comparison B). Bland Altman plots of agreement are shown in Fig. 2 and measures of repeatability and ICC values are listed in Tables 3 and 4. As shown in the Bland Altman plots, all measures of ventricular size, function, and mass were highly repeatable with narrow limits of agreement. Further, interstudy differences did not vary within the range of values studied for each parameter and there was no significant systematic bias. The 2 SD of the differences (repeatability coefficient) expressed as a percent of the population mean for each parameter were 10% 12% for EDV and EF for both LV and RV. However, repeatability coefficients were higher for ESV, SV, mass, and mass/volume ratio (18% 20%). Values for ICC were similar between LV and RV for most parameters except for ventricular mass, for which the ICC was lower for the RV (P ¼ 0.004). We also evaluated if interstudy variability was higher in patients with a transannular patch as compared with other surgical techniques and found that ICC values did not vary significantly for any of the ventricular parameters. Interaction with Interobserver Differences The results of interactions between interstudy variability and interobserver differences are summarized in Tables 3 and 4. In general, the limits of agreement were wider, repeatability coefficients were higher, bias was higher, and ICC coefficients were lower when Examinations 1 and 2 were interpreted by different readers rather than a single reader. This effect was seen with all parameters but was most pronounced for RV mass and mass-to-volume ratio. To further isolate the observer effect, we examined reproducibility when two blinded observers analyzed the same study on each patient (comparison A) and found that the repeatability coefficient was the highest for RV mass and mass-to-volume ratio. For these parameters, repeatability coefficients were reasonable (<30%) only when a single observer analyzed different studies. Systolic Versus Diastolic Assessment of Mass Given that interobserver repeatability coefficient was highest for assessment of ventricular mass at end-diastole, we measured mass at end-systole to assess if repeatability improved. Agreement was slightly higher for both LV and RV mass when measured at end-systole. For LV mass, the repeatability coefficient improved from 16.5 to 14 g and the ICC increased from to (P ¼ 0.002), when measurements were made at end-systole. For RV mass, the repeatability coefficient decreased from 15.8 to 13.8 g and ICC increased slightly from to (P ¼ 0.13), when measurements were made at end-systole. DISCUSSION Because of their association with long-term outcome, CMR parameters of ventricular size and function are used to guide management decisions in patients with repaired TOF. Although serial measurements are commonplace in clinical practice, little is known about the magnitude of variability in measurements from study to study that is related to differences in technical scan parameters and analysis by different readers independent of biological changes. Our results show that CMR parameters of left and right ventricular volumes, ejection fractions, and mass are highly repeatable from study to study when a single observer analyzes both studies. Assuming no biologic change from examination to examination, RVEDV values can be expected to vary up to 9% and variability in RVEF values may reach 12% relative to baseline measurements. Variability is somewhat higher for most parameters when different observers analyze sequential studies, with the impact being most prominent on measurements of RV mass and Table 2 CMR Ventricular Measurements (Mean 6 SD) Absolute measurement Indexed to BSA Left ventricle End-diastolic volume ml ml/m 2 End-systolic volume ml ml/m 2 Stroke volume ml ml/m 2 Ejection fraction % N/A Mass a g g/m 2 Mass-to-volume ratio N/A Right ventricle End-diastolic volume ml ml/m 2 End-systolic volume ml ml/m 2 Stroke volume ml ml/m 2 Ejection fraction % N/A Mass a g g/m 2 Mass-to-volume ratio N/A BSA, body surface area. Measurement during diastole.

4 832 Blalock et al. Figure 2. Bland Altman plots of the mean interstudy difference (comparison B) for RV and LV end-diastolic and end-systolic volumes, ejection fraction, stroke volume, and mass. For each parameter the average of the measurements from both studies is plotted on the x-axis and the interstudy difference is plotted on the y-axis. The red horizontal line plots the mean difference and the dashed blue lines indicated the limits of agreement (2 SD of the interstudy difference) for each parameter.

5 Variability in CMR measurements in TOF 833 Table 3 Interstudy Variability and Effect of Multiple Observers on LV Parameters 2SD of differences (repeatability coefficient) Bland-Altman analysis Repeatability coefficient as % of mean value for parameter Mean difference (bias) End-diastolic volume (ml) A. Interobserver, same examination % B. Interstudy, same observer % C. Multiple observers, 2 examinations 24 15% End-diastolic volume/bsa (ml/m 2 ) A. Interobserver, same examination 6.1 7% B. Interstudy, same observer % C. Multiple observers, 2 examinations % End-systolic volume (ml) A. Interobserver, same examination % B. Interstudy, same observer % C. Multiple observers, 2 examinations % End-systolic volume/bsa (ml/m 2 ) A. Interobserver, same examination % B. Interstudy, same observer % C. Multiple observers, 2 examinations % Ejection fraction (%) A. Interobserver, same examination % B. Interstudy, same observer % C. Multiple observers, 2 examinations % Mass during diastole (g) A. Interobserver, same examination % B. Interstudy, same observer % C. Multiple observers, 2 examinations % Mass during diastole/bsa (g/m 2 ) A. Interobserver, same examination % B. Interstudy, same observer % C. Multiple observers, 2 examinations % Mass-to-volume ratio A. Interobserver, same examination % B. Interstudy, same observer % C. Multiple observers, 2 examinations % BSA, body surface area; ICC, intraclass correlation coefficient; SD, standard deviation. ICC mass-to-volume ratio. With different observers, assuming no systematic bias, RVEDV values can be expected to vary up to 13% and variability in RVEF values may reach 21% of the baseline values. In case of systematic measurement bias between observers, these limits may be wider. Aside from parameters related to ventricular mass, the repeatability of RV measurements does not differ significantly from that of LV measurements. Of note, we cross-referenced the long-axis and short-axis images for all analyses to help define the basal slice of both ventricles, as recommended (6). Several groups have reported on intra- and interobserver variability of CMR ventricular measurements in TOF patients (6,13,14). These studies focused on the variance related to repeated analysis of a single CMR examination by a single or two observers. However, in clinical practice, additional variance may be introduced during serial follow-up of patients due to differences in scan planes and other technical parameters. Data on this additional source of variability are limited. Maroules et al (15) studied healthy volunteers and a small group of adults with noncongenital heart disease and found measurements of LV and RV to be highly repeatable. Similar results were reported by Grothues et al (16) using an older gradient echo imaging sequence. However, neither study included patients with dilated or abnormal RVs, which limited their generalizability. Our study is the first to evaluate interstudy variability in ventricular parameter of TOF patients using contemporary imaging techniques. The limits of interstudy variability for RV parameters in our study are comparable to those reported by Maroules et al (15) in patients with acquired LV disease, suggesting that RV dilation and prior surgical repair do not introduce additional variability in measurements. Identification of ventricular borders can be difficult in the presence of a transannular patch and this may be an additional source of interobserver variability. Nevertheless, we found that interstudy variability for RV parameters was not influenced by the presence of a transannular patch. Similar to prior studies (15), the limits of agreement were slightly wider for RV parameters in comparison to LV parameters. However, ICC values were not statistically different for most parameters except ventricular mass. Agreement limits for RV mass were significantly wider compared to other parameters.

6 834 Blalock et al. Table 4 Interstudy Variability and Effect of Multiple Observers on RV Parameters 2SD of differences (repeatability coefficient) Bland-Altman analysis Repeatability coefficient as % of mean value for parameter Mean difference (bias) End-diastolic volume (ml) A. Interobserver, same examination % B. Interstudy, same observer % C. Multiple observers, 2 examinations % End-diastolic volume/bsa (ml/m 2 ) A. Interobserver, same examination % B. Interstudy, same observer % C. Multiple observers, 2 examinations % End-systolic volume (ml) A. Interobserver, same examination % B. Interstudy, same observer % C. Multiple observers, 2 examinations % End-systolic volume/bsa (ml/m 2 ) A. Interobserver, same examination % B. Interstudy, same observer % C. Multiple observers, 2 examinations % Ejection fraction (%) A. Interobserver, same examination % B. Interstudy, same observer % C. Multiple observers, 2 examinations % Mass during diastole (g) A. Interobserver, same examination % B. Interstudy, same observer % C. Multiple observers, 2 examinations % Mass during diastole/bsa (g/m 2 ) A. Interobserver, same examination % B. Interstudy, same observer % C. Multiple observers, 2 examinations % Mass-to-volume ratio A. Interobserver, same examination % B. Interstudy, same observer % C. Multiple observers, 2 examinations % BSA, body surface area; ICC, intraclass correlation coefficient; SD, standard deviation. ICC Importantly, agreement was highest when a single observer analyzed repeat examinations, suggesting that most of the variance was related to interobserver differences in contouring. Difficulty in visualizing the thin RV wall likely impacts this measurement. Indeed, we found that interobserver agreement was slightly improved when RV mass was measured in systole instead of diastole. Several limitations of this study are worth noting. The relatively small sample size may obscure minor differences between RV and LV measurements of repeatability. Exclusion of young children and infants, who could not be included due to ethical considerations with prolongation of anesthesia, limits the interpretation to older patients. However, this is the group that commonly undergoes serial CMR examinations. All analyses were performed on ventricular short-axis images and the variability in analyzing axial images was not investigated. Finally, studies were performed on the same scanner and hence variability related to vendor type was not assessed. In conclusion, most CMR measures of ventricular size and function have acceptable repeatability in adolescents and adult patients with repaired TOF. Measurements of RV mass are subject to higher variability, which is slightly improved when measured during systole. For most parameters, agreement limits are wider when measurements are performed by multiple operators. These results will aid in the interpretation of study-to-study variations in individual patients during serial clinical follow-up and will facilitate the estimation of sample size for future clinical trials in TOF patients. REFERENCES 1. Geva T, Sandweiss BM, Gauvreau K, Lock JE, Powell AJ. Factors associated with impaired clinical status in long-term survivors of tetralogy of Fallot repair evaluated by magnetic resonance imaging. J Am Coll Cardiol 2004;43: Khairy P, Aboulhosn J, Gurvitz MZ, et al. Arrhythmia burden in adults with surgically repaired tetralogy of Fallot: a multi-institutional study. Circulation 2010;122: Murphy JG, Gersh BJ, Mair DD, et al. Long-term outcome in patients undergoing surgical repair of tetralogy of Fallot. N Engl J Med 1993;329: Nollert G, Fischlein T, Bouterwek S, Bohmer C, Klinner W, Reichart B. Long-term survival in patients with repair of tetralogy of Fallot: 36-year follow-up of 490 survivors of the first year after surgical repair. J Am Coll Cardiol 1997;30:

7 Variability in CMR measurements in TOF Ghai A, Silversides C, Harris L, Webb GD, Siu SC, Therrien J. Left ventricular dysfunction is a risk factor for sudden cardiac death in adults late after repair of tetralogy of Fallot. J Am Coll Cardiol 2002;40: Mooij CF, de Wit CJ, Graham DA, Powell AJ, Geva T. Reproducibility of MRI measurements of right ventricular size and function in patients with normal and dilated ventricles. J Magn Reson Imaging 2008;28: Walsh R, Salem Y, Shah A, Lai WW, Nielsen JC. Repeatability of cardiac-mri-measured right ventricular size and function in congenital heart disease. Pediatr Radiol 2011;41: Geva T. Repaired tetralogy of Fallot: the roles of cardiovascular magnetic resonance in evaluating pathophysiology and for pulmonary valve replacement decision support. J Cardiovasc Magn Reson 2011;13:9. 9. Samyn MM, Powell AJ, Garg R, Sena L, Geva T. Range of ventricular dimensions and function by steady-state free precession cine MRI in repaired tetralogy of Fallot: right ventricular outflow tract patch vs. conduit repair. J Magn Reson Imaging 2007;26: Alfakih K, Plein S, Thiele H, Jones T, Ridgway JP, Sivananthan MU. Normal human left and right ventricular dimensions for MRI as assessed by turbo gradient echo and steady-state free precession imaging sequences. J Magn Reson Imaging 2003;17: Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986;1: Donner A. A review of inference procedures for the intraclass correlation coefficient in a one-way random effects model. Int Stat Rev 1986;54: Hudsmith LE, Petersen SE, Francis JM, Robson MD, Neubauer S. Normal human left and right ventricular and left atrial dimensions using steady state free precession magnetic resonance imaging. J Cardiovasc Magn Reson 2005;7: Moon JC, Lorenz CH, Francis JM, Smith GC, Pennell DJ. Breathhold FLASH and FISP cardiovascular MR imaging: left ventricular volume differences and reproducibility. Radiology 2002;223: Maroules CD, McColl R, Khera A, Peshock RM. Interstudy reproducibility of SSFP cine magnetic resonance: impact of magnetic field strength and parallel imaging. J Magn Reson Imaging 2008; 27: Grothues F, Moon JC, Bellenger NG, Smith GS, Klein HU, Pennell DJ. Interstudy reproducibility of right ventricular volumes, function, and mass with cardiovascular magnetic resonance. Am Heart J 2004;147: