Impact of audit of routine second-trimester cardiac images using a novel image-scoring method

Transcription

1 Ultrasound Obstet Gynecol 29; 33: Published online 9 April 29 in Wiley InterScience ( DOI: 1.12/uog.6323 Impact of audit of routine second-trimester cardiac images using a novel image-scoring method S. SAIRAM*, A. M. A. AWADH*, K. COOK*, A. T. PAPAGEORGHIOU* and J. S. CARVALHO* *Fetal Medicine Unit, St George s Healthcare NHS Trust and Brompton Fetal Cardiology, Royal Brompton Hospital, London, UK KEYWORDS: audit; cardiac; echocardiography; fetus ABSTRACT Objective To assess the impact of using an objective scoring method to audit cardiac images obtained as part of the routine week anomaly scan. Methods A prospective audit and re-audit (6 months later) were conducted on cardiac images obtained by sonographers during the routine anomaly scan. A new image-scoring method was devised based on expected features in the four-chamber and outflow tract views. For each patient, scores were awarded for documentation and quality of individual views. These were called Documentation Scores and View Scores and were added to give a Patient Score which represented the quality of screening provided by the sonographer for that particular patient (maximum score, 15). In order to assess the overall performance of sonographers, an Audit Score was calculated for each by averaging his or her Patient Scores. In addition, to assess each sonographer s performance in relation to particular aspects of the various views, each was given their own View Scores, derived from image documentation and details of four-chamber view (magnification, valve offset and septum) and left and right outflow tract views. All images were scored by two reviewers, jointly in the primary audit and independently in the re-audit. The scores from primary and re-audit were compared to assess the impact of feedback from the primary audit. Results Eight sonographers participated in the study. The median Audit Score increased significantly (P <.1), from (range, ) in the primary audit to 12.4 (range, 13.6) in the re-audit. Scores allocated by the two reviewers in the re-audit were not significantly different (P =.8). Conclusion Objective scoring of fetal heart images is feasible and has a positive impact on the quality of cardiac images acquired at the time of the routine anomaly scan. This audit tool has the potential to be applied in every obstetric scanning unit and may improve the effectiveness of screening for congenital heart defects. Copyright 29 ISUOG. Published by John Wiley & Sons, Ltd. INTRODUCTION Congenital heart disease (CHD) is a leading cause of infant mortality, with an incidence estimated at about 4 13 per 1 live births 1 3. These anomalies are frequently missed on prenatal ultrasonography 4,5.Withimprovementsinthe quality of ultrasound equipment and training programs, detection rates have improved, but they still vary widely from center to center 6 8. The training of operators performing routine screening of the fetal heart to include basic and extended basic cardiac views is designed to maximize the detection of heart anomalies during a second-trimester scan 8 1. Following training, there is a need, however, to maintain the standards expected of operators and thus the units in which they work. Ongoing quality assurance is in place to ensure that sonographers maintain the standards required in the measurement of fetal nuchal translucency thickness in the week scan However, as yet, there has been no such attempt to assess the quality of cardiac images obtained at the time of screening. The aims of this study were to assess the feasibility of using an imagescoring method to audit the quality of cardiac images and to assess the impact of such objective scoring on the images obtained 6 months later. METHODS This was a prospective study carried out in a teaching hospital. In our institution, all women are offered an ultrasound examination at 11 to weeks Correspondence to: Dr S. Sairam, Fetal Medicine Unit, St George s Hospital, Blackshaw Road, London SW17 QT, UK ( shanthi.sairam@gmail.com) Accepted: 9 September 28 Copyright 29 ISUOG. Published by John Wiley & Sons, Ltd. ORIGINAL PAPER

2 546 Sairam et al. of gestation for pregnancy dating and screening for fetal chromosomal abnormalities by nuchal translucency measurement. A routine fetal anomaly scan is then offered at weeks of gestation, when screening for CHD is performed. At the time of the audit, this was done by obtaining a four-chamber view and, whenever possible, views of the outflow tracts. The audit cycle was completed with feedback given to every sonographer and re-audit was planned for 6 months later. Primary audit Participants included sonographers performing routine screening. During a pre-defined audit week, they were asked to submit a set of thermal images (STI) of the views obtained routinely for cardiac screening, for every fetal anomaly scan performed. No specifications were given regarding the number of images to be submitted or which features were to be demonstrated. In addition to the routine views using gray-scale ultrasound, some sonographers also submitted images using color Doppler. As this would not be part of routine screening, we only included the gray-scale images for the purposes of this study. A new image-scoring method was devised to evaluate images of the standard views used for routine cardiac screening at the week scan (Figure 1). Scores were awarded both for documentation of the cardiac views ( Documentation Score ) and for the quality of information in each view ( View Score ). The Documentation Score ranged from to 4, with 1 point being given for the imaging of each of four different cardiac views, namely, situs view, four-chamber view and right (RVOT) and left (LVOT) ventricular outflow tract views (Table 1; Score = 1 for image(s) provided and Score = for image not provided). The Documentation Score for a patient therefore could not be increased simply by providing multiple images of one cardiac view. Images depicting the fetal situs or fetal laterality, i.e. a crosssection of the fetal abdomen showing the position of the fetal stomach, were scored as part of the documentation as they were indicative of the sonographer s attempt to ascertain the fetal left side. The View Score ranged from to 11 and comprised a more detailed analysis of the fourchamber and outflow tract views, scored for the quality of their information as described in Table 1. The three vessels view (3VV) was considered as equivalent to the RVOT view and these two views are used interchangeably for the purposes of this paper. The sum of the Documentation and View Scores for every STI produced a Patient Score, which gave an idea of the quality of screening provided by the sonographer for a particular patient (maximum Patient Score = 15). In order to assess the overall performance of each sonographer, an Audit Score was calculated by averaging his or her Patient Scores. In addition, to assess each sonographer s performance in relation to particular aspects of the various views, six View Scores were derived, one each for documentation of views, four-chamber details (magnification, atrioventricular valve offset and septum), LVOT and RVOT/3VV. These View Scores were averages of the individual View Scores obtained by a particular sonographer across all of his or her patients. The various scores aresummarizedintable2. Each STI was scored by two reviewers jointly (S.S. and A.A.); the sonographers were unaware of the scoring method. Each sonographer was given detailed individual feedback on their performance, based on these scores. They were given insight into what information was expected in each view. In addition, they were encouraged to observe S.S. or J.S.C. carrying out fetal echocardiography. Re-audit Re-audit was carried out 6 months after the primary audit without the sonographers knowledge, during a week chosen arbitrarily. For each sonographer, gray-scale cardiac images from five consecutive anomaly scans were retrieved from the electronic database used for archiving Figure 1 (a) Four-chamber view showing adequate magnification (Score 2) with good demonstration of atrioventricular valve offset (Score 2) and intact ventricular septum (Score 2). (b) View of the left ventricular outflow tract clearly demonstrating the septoaortic continuity and the vessel arising from the left ventricle (Score 2). (c) Three vessels view showing all three vessels (pulmonary artery, aorta and superior vena cava), their relative sizes and position (Score 2). Copyright 29 ISUOG. Published by John Wiley & Sons, Ltd. Ultrasound Obstet Gynecol 29; 33:

3 Audit of cardiac images 547 Table 1 Criteria for score-based objective evaluation of routine second-trimester cardiac images Score type Score Documentation Score Situs 1 Four-chamber view (one or more images) 1 Left ventricular outflow tract view 1 Right ventricular outflow tract and/or three vessels view 1 View Scores Four-chamber view Magnification Heart image fills more than one third of display screen (only heart seen no chest wall) 3 Heart image fills one third of the display screen (heart and entire chest seen, but nothing around the chest) 2 Heart image fills less than one third of the display screen (entire chest wall and some area around) 1 Heart and chest with a lot of surrounding area Atrioventricular valve offset Offset seen clearly 2 Offset demonstrated poorly 1 Offset not seen at all Interventricular septum Clear, well-defined septum (apex to crux) 2 Septum unclear with areas suggestive of drop-out 1 Septum defined poorly with definite drop-out Left ventricular outflow tract view Septoaortic continuity demonstrated clearly with outflow tract 2 Septoaortic continuity and outflow tract seen with difficulty 1 Poor or no demonstration of septoaortic continuity or outflow tract Right ventricular outflow tract and/or three vessels view Clear and well demonstrated 2 Unclear but just demonstrated 1 Poor or no demonstration Patient Score 15 (maximum) Table 2 Definition and significance of the scores used in the objective assessment of routine second-trimester cardiac images Score Definition Documentation Score Indicates which views were submitted for audit by a particular sonographer per patient (maximum = 4, regardless of number of images for a particular view) View Score Indicates anatomical details demonstrated in each particular view submitted per patient by a particular sonographer (maximum = 11) Patient Score Indicates quality of screening performed by a particular sonographer on a particular patient. Derived from sum of Documentation and View Scores (maximum = 15). Audit Score Provides overview of overall performance of a particular sonographer. Derived by averaging all Patient Scores obtained by a sonographer. Allows comparison between sonographers and assessment of change in each sonographer s performance after subsequent re-audit. View Score Provides in-depth assessment of performance of a particular sonographer in documenting and demonstrating each view. Derived by averaging separately Documentation Scores and View Scores for individual views obtained by a sonographer across all his/her patients. Allows detailed individual feedback on performance. images in the unit and a set of digital images (SDI) was compiled on a computer. The scoring was done using the same scoring system as for the primary audit but this time by two independent observers (S.S. and K.C.) so as to assess interobserver variability. The average of the two observers scores was used for comparison between primary and re-audit. In addition, information relating to the maternal body mass index (BMI) and the machine used (Philips ATL HDI 5, Philips Healthcare, Best, The Netherlands; Aloka Prosound4/SSD55, Aloka Co Ltd, Tokyo, Japan) was collected in order to examine whether these factors influenced the scores. The latter was particularly relevant, as the sonographers did not use the same machine all the time. The scores from primary and re-audit were compared to assess the impact of feedback from the primary audit. Audit Scores and Patient Scores were compared to provide an overview of the sonographers overall performance. In addition, View Scores were compared to assess the impact on specific views. Patient scores were correlated with BMI and the ultrasound machines used. Interobserver variability was assessed by comparing the Copyright 29 ISUOG. Published by John Wiley & Sons, Ltd. Ultrasound Obstet Gynecol 29; 33:

4 548 Sairam et al. View Scores awarded independently by the two observers in the re-audit. Statistics The Mann Whitney U-test was used to compare Patient Scores between the primary audit and the re-audit. The Wilcoxon signed-rank test was used to compare the Audit and View Scores between the primary and the re-audit. Pearson s correlation coefficient was used to assess the strength of the relationship between the Patient Scores and BMI. The Mann Whitney U-test was used to assess the strength of the relationship between the Patient Scores and the ultrasound machine used. Interobserver variability was assessed by applying Friedman s tests for non-parametric data on the re-audit View Scores 15,16. RESULTS Eight sonographers from the primary audit were available for re-audit and only data from these eight sonographers were included in this study. On average, nine STIs (range, 4 18) were scored per sonographer in the primary audit and five SDIs were scored in the re-audit. The median Patient Score at the first audit was 11 (range, 7 14). This showed a significant improvement at the re-audit to 12.5 (range, ); (P <.1). The median Audit Score at the primary audit was (range, ). This also showed a significant improvement at the reaudit, increasing to 12.4 (range, 13.6; P <.1) (Figure 2). The View Scores, which were used to provide detailed feedback, allowed analysis of the strengths and weaknesses of every individual participant, providing information about the areas that needed focusing on and, when required, training. The impact of this feedback on all views can be seen in Figure 3. More detailed analysis with a breakdown of these scores for every view is given in Figure 4. The Patient Scores from the re-audit showed an apparent negative correlation with the maternal BMI, but 15 Average View Scores Max = 4 Documentation Max = 3 4CV mag 4CV offset 4CV septum Max = 2 LVOT RVOT/3VV Figure 3 Average View Scores in primary audit ( ) and re-audit 6 months later ( ) of routine second-trimester cardiac images. Maximum scores possible are indicated (Max). View Scores were derived from image Documentation Scores and from details of four-chamber view (4CV) (magnification (mag), valve offset and septum) and left and right outflow tract views (LVOT and RVOT/3VV). this was not statistically significant (r =.2162; 95% CI,.491 to.97). There was no statistically significant difference between Patient Scores obtained from the two sets of ultrasound machines (Philips ATL HDI 5 and Aloka Prosound4/SSD55) (P =.496). Friedman s test showed no statistically significant difference between the scores allocated by each of the two reviewers in the re-audit (P =.8). DISCUSSION We have presented an objective image-scoring method for auditing cardiac images in a fair and constructive manner. This audit tool is the first of its kind and may help in standardizing the cardiac images obtained during screening for CHD at the time of the routine anomaly scan. Implementation of this audit led to a significant improvement in the sonographers median Audit Score. This improvement was due to an increase in the median Patient Score, reflecting better documentation and higher quality of images during screening. Audit Score Figure 2 Individual sonographers Audit Scores in primary audit ( ) and re-audit 6 months later ( )ofroutine second-trimester cardiac images. Feasibility of conducting an audit It is evident from this audit that it is possible to assess objectively the quality of still images obtained for routine screening for CHD. It is well accepted that the fetal heart anatomy is best viewed in dynamic scanning mode and cannot always be well-represented with still images. However, storing dynamic images during routine screening is not always practical, whereas many obstetric ultrasound departments may already have facilities for printing thermal images. Therefore, examination of still images may at present be the only practical option for quality assessment. Furthermore, the process of obtaining still images of good quality means that good quality dynamic imaging is undertaken. This audit proved to be both constructive Copyright 29 ISUOG. Published by John Wiley & Sons, Ltd. Ultrasound Obstet Gynecol 29; 33:

5 Audit of cardiac images 549 (a) Documentation Score P <.5 (b) four-chamber view magnification score NS (c) four-chamber view offset score P <.25 (d) four-chamber view septum score NS.2.2 (e) 2. P <.1 (f) 2. NS LVOT score three vessels view score Figure 4 Individual View Scores in primary audit ( ) and re-audit 6 months later ( ) of routine second-trimester cardiac images along with corresponding P-values for comparison between primary and re-audits. (a) Documentation Scores; (b d) details of four-chamber view; (e) left ventricular outflow tract (LVOT); (f) three vessels view. NS, not significant. and feasible. With the feedback provided, sonographers gained a better understanding of what they were expected to demonstrate in each view as well as the standard of images required. This audit complements the guidelines issued by the International Society of Ultrasound in Obstetrics & Gynecology Education Committee. Formal auditing of effectiveness of screening for CHD involves obtaining outcomes for each pregnancy screened. The length of infant follow-up required for such audit coupled with the relatively low incidence of the disease makes this difficult in day-to-day practice. In comparison, auditing images obtained by sonographers performing screening is easier and can be an ongoing process of audit, feedback and re-audit. This could therefore be the first step in assessing the quality of screening and a tool for improving the effectiveness of this screening 1. Copyright 29 ISUOG. Published by John Wiley & Sons, Ltd. Ultrasound Obstet Gynecol 29; 33:

6 55 Sairam et al. Audit Score This score, obtained from all images a particular sonographer obtained when screening various patients, gave an overall assessment of his or her screening standard. Given that the sonographers were not aware of the scoring method in the primary audit, it is commendable that seven of the eight sonographers achieved 65% of the maximum score (> 1/15). The wide range in the Audit Scores ( ) amongst sonographers in the same department highlights the varied ability that may be encountered within the same unit. The improvement in their scores in the re-audit, with 1% of them scoring above 65%, along with the significant improvement in the median score, is proof of the positive impact of this audit. This was not a short-term improvement, as is often seen in studies before and after training, because 6 months had elapsed from the primary audit. Furthermore, the improvement is likely to have been a real effect, because sonographers were not aware of the time of reaudit. We believe that this improvement is attributable to the individualized feedback that was provided following the primary audit, which essentially educated each sonographer on the important, specific aspects of the scan on which they needed to concentrate. Documentation and View Scores At the time of the audit, the protocol in our department indicated that screening the four-chamber view of the heart is mandatory, whereas screening the outflow tracts would be desirable. Despite this, all sonographers had screened the outflow tracts and had provided images to demonstrate them, as indicated by their scores for documentation. Mandatory screening of outflow tracts has since been included in our unit s protocol. Ascertaining fetal laterality during routine screening was also considered desirable at the time of the primary audit and most sonographers routinely looked at visceral situs and cardiac position but did not necessarily take an image of this. Incorporating this into the documentation increased the awareness of the fetal situs and the importance of determining the relative cardiac position at the time of screening. We plan to educate sonographers further, to acquire the upper mediastinal view, which not only demonstrates the three vessels, with the aortic and ductal arches, but also demonstrates their position in relation to the fetal trachea. In future audits, documenting this view and assessing the quality of the information provided will be incorporated into the scoring system. Analyzing the View Scores obtained by each sonographer allowed the reviewers to break down and highlight the areas which required particular attention on an individual basis. Even in the presence of good Patient Scores, it is possible that a sonographer may be underperforming regularly in a particular view. For example, although H scored well in both audits, there was no improvement in the overall score (Audit Score). A detailed analysis of the View Scores, however, showed an improvement in outflow tract scores but a loss in some aspects of the four-chamber view scores, both of which were discussed individually with this sonographer. If consistent low scores are identified for a specific view, this can be addressed and efforts made to overcome the difficulty the sonographer may be facing in obtaining a particular view. It is evident from Figure 3 that scores increased for all views except for the four-chamber magnification. The reduction in this score value between the two audits must be interpreted with caution as it is just as important to assess the relative size of the heart and chest (score closer to 2) as it is to demonstrate the intracardiac features (score closer to 3). Figure 4b shows that in both audits, six of the individual sonographers scores for magnification were 2. We believe that magnification corresponding to a score of 2 as described in the Table 1 is perhaps adequate for screening. Alternatively, two images may be used one to assess the cardiothoracic ratio and the second to demonstrate the cardiac features in greater detail. The vast majority of sonographers showed improvement in most View Scores (Figure 4), although statistical significance was only seen for three of the six scores analyzed. This may have been due to the relatively small numbers of sonographers in this study as well as to the relatively high score values already obtained in the primary audit. The participants were all qualified sonographers. While their level of experience varied, all were considered to be at the level of Superintendent Grade III. The two audits were 6 months apart and during this time, no specific change was made in the unit in terms of work, supervision or further training. Opportunities to observe specialist fetal echocardiography have always been available in the unit and this was no different during these 6 months. However, as part of the feedback from the audit, sonographers were encouraged to attend more of these sessions. It would be difficult to attribute the improvement in scores to anything other than the awareness created by the audit itself, as indicated by the sonographers themselves following the feedback from the primary audit. The sonographers were all capable of achieving high scores and this audit provided them with the direction and guidelines needed for a higher level of performance. We have not included the comments from individual sonographers in this manuscript as this would make an article in itself. BMI and equipment It is likely that BMI plays a significant role in the quality of views obtained for cardiac screening. Although there was a negative correlation between increasing BMI and quality of views in the re-audit, this was not statistically significant, which may be due to a limited power to detect this correlation in our study. When assessing the effect of equipment, there was no correlation between scores and the machines used. This may be due to the fact that equipment in our unit is regularly upgraded, meaning that differences in quality between machines are small. Copyright 29 ISUOG. Published by John Wiley & Sons, Ltd. Ultrasound Obstet Gynecol 29; 33:

7 Audit of cardiac images 551 CONCLUSION We have demonstrated that it is possible to audit the cardiac images obtained at the routine anomaly scan in an objective manner using a simple and universally applicable scoring method. The scoring can be modified to suit local needs and protocols. For instance, in a unit that uses only the four-chamber view for cardiac screening, the audit can be carried out using the scores for the four-chamber view alone. The unit could then introduce outflow tract views and audit these at a later date. Such an approach, on a wider scale, would undoubtedly improve overall standards in routine fetal cardiac ultrasound examinations. This objective scoring system allows individual feedback to be provided, highlighting areas that require specific attention in the views obtained for cardiac screening. It can be used to audit sonographers and to assess trainees, as those features that are expected in the standard basic and extended cardiac views can be clearly formulated. We found that application of an audit cycle, using primary audit, feedback and re-audit, led to a significant improvement in scores obtained and that this scoring and feedback improved the process of care. Assessment of this on a larger scale needs to be carried out, to determine whether it is likely to result in an improvement in patient care, i.e. in improved detection rates for CHD. Finally, the availability of such a cardiac scoring method may make certification in cardiac scanning a possibility for the future. REFERENCES 1. Ferencz C, Rubin JD, McCarter RJ, Brenner JL, Neill CA, Perry LW, Hepner SI, Downing JW. Congenital heart disease: prevalence at live birth. The Baltimore-Washington infant study. Am J Epidemiol 1985; 121: Hoffman JIE. Incidence of congenital heart disease: I postnatal incidence, Pediatr Cardiol 1995; 16: Cuneo BF, Curran LF, Davis N, Elrad H. Trends in prenatal diagnosis of critical cardiac defects in an integrated obstetric and pediatric cardiac imaging center. J Perinatol 24; 24: Bull C. Current and potential impact of fetal diagnosis on prevalence and spectrum of serious congenital heart disease at term in the UK. Lancet 1999; 354: Abu-Harb M, Hey E, Wren C. Death in infancy from unrecognized congenital heart disease. Arch Dis Child 1994; 71: Nelson NL, Filly RA, Goldstein RB, Callen PW. The AIUM/ ACR antepartum obstetrical sonographic guidelines: expectations for detection of anomalies. J Ultrasound Med 1993; 12: Simpson LL. Screening for congenital heart disease. Obstet Gynecol Clin North Am 24; 31: Hunter S, Heads A, Wylie J, Robson S. Prenatal diagnosis of congenital heart disease in the northern region of England: benefits of a training programme for obstetric ultrasonographers. Heart 2; 84: Friedman AH, Kleinman CS, Copel JA. Diagnosis of cardiac defects: where we ve been, where we are and where we re going. Prenat Diagn 22; 22: Cardiac screening examination of the fetus: guidelines for performing the basic and extended basic cardiac scan. International Society of Ultrasound in Obstetrics & Gynecology. Ultrasound Obstet Gynecol 26; 27: Herman A, Maymon R, Dreazen E, Caspi E, Bukovsky I, Weinraub Z. Nuchal translucency audit: a novel image scoring method. Ultrasound Obstet Gynecol 1998; 12: Snijders RJ, Noble P, Sebire N, Souka A, Nicolaides KH. UK multicentre project on assessment of risk of trisomy 21 by maternal age and fetal nuchal-translucency thickness at 1 14 weeks of gestation. Fetal Medicine Foundation First Trimester Screening Group. Lancet 1998; 352: Snijders RJ, Thom EA, Zachary JM, Platt LD, Greene N, Jackson LG, Sabbagha RE, Filkins K, Silver RK, Hogge WA, Ginsberg NA, Beverley S, Morgan P, Blum K, Cilis P, Hill LM, Hecker J, Wapner RJ. First-trimester trisomy screening: nuchal translucency measurement training and quality assurance to correct and unify technique. Ultrasound Obstet Gynecol 22; 19: Dudley NJ, Chapman E. The importance of quality management in fetal measurement. Ultrasound Obstet Gynecol 22; 19: Salomon LJ, Bernard JP, Duyme M, Doris B, Mas N, Ville Y. Feasibility and reproducibility of an image-scoring method for quality control of fetal biometry in the second trimester. Ultrasound Obstet Gynecol 26; 27: Fries N, Althuser M, Fontanges M, Talmant C, Jouk PS, Tindel M, Duyme M. Quality control of an image-scoring method for nuchal translucency ultrasonography. Am J Obstet Gynecol 27; 196: 272.e1 272.e5. Copyright 29 ISUOG. Published by John Wiley & Sons, Ltd. Ultrasound Obstet Gynecol 29; 33: