Improved laparoscopic skills in gynaecology trainees following a simulation- training program using takehome box trainers

Transcription

1 Aust N Z J Obstet Gynaecol 2019; 59: DOI: /ajo ORIGINAL ARTICLE Improved laparoscopic skills in gynaecology trainees following a simulation- training program using takehome box trainers Erin Wilson 1,2, Sarah Janssens 2,3, Lucas A. McLindon 2,3, David G. Hewett 2,3, Brian Jolly 4 and Michael Beckmann 1,2,3 1 Mater Research, University of Queensland, South Brisbane, Queensland, Australia 2 University of Queensland Faculty of Medicine, Herston, Queensland, Australia 3 Mater Health, South Brisbane, Queensland, Australia 4 School of Medicine and Public Health, University of Newcastle, Newcastle, New South Wales, Australia Correspondence: Dr Erin Wilson, Mater Research, Level 3, Aubigny Place, Rayond Terrace, South Brisbane, QLD 4101, Australia. erin.wilson1@uqconnect.edu.au Present addresses: David G. Hewett, UQ School of Medicine, Mayne Medical School, Herston, Queensland, Australia Brian Jolly, University of Newcastle, Callaghan, New South Wales, Australia Conflicts of Interest: The authors report no conflicts of interest. Received: 30 August 2017; Accepted: 16 February 2018 Background: Lack of time and access to equipment are recognised barriers to simulation training. Aim: To investigate the effect of a take- home laparoscopic simulator training program on the laparoscopic skills of gynaecology trainees. Method: Participants (n = 17 in 2015, n = 16 in 2016) were supplied with a box trainer, associated equipment and instructions on self- directed training. A program was designed and implemented in 2015 comprising of ten weekly laparoscopic skills tasks and modified in 2016 to eight monthly tasks. Half of the participants were randomly allocated a supervisor. Participants performed baseline and post- training assessments of laparoscopic skills in a box trainer task (thread transfer) and virtual reality simulator tasks (laparoscopic tubal ligation and bilateral oophorectomy). Results: Trainees in 2015 demonstrated an improvement in the median time to complete the laparoscopic tubal ligation task (baseline 124 s vs post- training 91 s, P = 0.041). There was no difference in the number of tubal ligation bleeding incidents, or in the time taken to complete the box trainer thread transfer task. In 2016 trainees demonstrated improvement in tubal ligation time (baseline 251 vs 71 post- training, P = 0.021) and bilateral oophorectomy time (baseline 891 s vs 504 post- training, P = 0.025). There was no significant difference in other outcome measures. There was no difference found in performance when groups were compared by supervisor allocation. Conclusion: A take- home box trainer simulation- training program was associated with improvement in laparoscopic skills. This type of program may improve trainee access to simulation training. KEYWORDS graduate medical education [MeSH], gynaecology, laparoscopy, simulation, surgery, operative/education [MeSH] The Royal Australian and New Zealand College of Obstetricians and Gynaecologists wileyonlinelibrary.com/journal/anzjog

2 E. Wilson et al. 111 INTRODUCTION There is a reported lack of confidence among gynaecology graduates to perform the breadth of procedures in the specialty. 1,2 The reduced number of real- procedure opportunities for teaching and practise has hampered surgical training in gynaecology. 3,4 Simulation allows additional opportunities for surgical training and is being increasingly used as an adjunct to traditional methods. 5 It has been proposed that the acquisition of expertise comes from deliberate practice as opposed to other assumed factors such as experience and innate ability. 6 Simulation allows deliberate practice (the repeated performance of an individual task to improve an aspect of overall performance); an activity not afforded by live operation exposure. 7 Simulation allows trainees to perform surgical tasks repeatedly in a controlled environment, which aids progress to stages of higher automaticity. 7 While evidence demonstrates the potential effectiveness of simulation, 8 11 there is a lack of uptake in surgical curricula in Australia and New Zealand 12 and internationally. 13 In order to improve the uptake of simulation training, the need for formal curricula has been suggested. 2,14 Despite advancements in simulation technology, progress in curriculum development lags. 15 There is no consistent approach to skill selection, timing of training, feedback or assessments Other barriers to simulation uptake include perceived inaccessibility of equipment and a lack of time. 12,19 21 There is a need to explore novel programs for simulation to increase trainee access to additional surgical training and improve their surgical skills. 22 Previous studies using take- home box trainers have produced variable results in terms of trainee engagement and skill improvement and further research into their role has been suggested. 22,23 It is also unclear whether supervision is essential for trainees in such programs, with literature providing conflicting evidence for such a requirement. 22,24,25 The desire to improve uptake of simulation prompted the design of a laparoscopic training program. The program incorporated take- home box trainers in order to improve trainee access to surgical simulation. The aim of this study was to determine the effectiveness of this curriculum on trainee skill development by investigating gynaecology trainee laparoscopic skills in simulated tasks following participation in the program. MATERIALS AND METHODS HREC/14/MHS/184) and the University of Queensland Ethics Review Committee (Approval Number: ). Participants All trainees in Obstetrics and Gynaecology (including RANZCOG trainees and non- accredited registrars) were invited to participate in the study. Half of the participants were randomly assigned a supervisor for the duration of the program. Randomisation occurred through the issuing of identical portable hard drives in identical unlabelled envelopes containing the program instructions and supervisor allocation as participants presented to collect their equipment. Supervisors were laparoscopic gynaecology surgeons at the Mater Hospital and participants who received supervisor allocation were encouraged to a video of their performance at each training task to their supervisor. Supervisors reviewed the videos (of usually less than three minutes duration) and provided written feedback and encouragement regarding technique enhancement via . The control group did not receive any feedback or supervision of their training during the program. Program features Following consent, participants were supplied with a take- home portable laparoscopic box trainer (eosim ProTrac model 18 ), associated instruments and software (and instrument tracking capability), a set of tasks with instructional videos and target performance levels (provided by eosim), a spreadsheet training logbook and program overview. Use of the equipment was demonstrated in an induction session and s were sent to remind participants of each training task throughout the program. In 2015, participants were instructed to complete a program of ten weekly laparoscopic tasks. The tasks included: thread transfer precision cutting paper clip untangle dice stacking paper fold glove tip capping precision suture placement thread pull horizontal suture intracorporeal suture and tie. A pragmatic, mixed methods study was undertaken at a tertiary hospital in 2015 and In 2015, a pilot program was run over a ten- week period. In response to participant feedback, modifications were made for a longer program in 2016, which ran over eight months. The study was deemed to meet the requirements for low/negligible risk research by the Mater Health Services Human Research Ethics Committee (Reference Number: In 2016, in response to participant feedback that there was not enough time to complete each training task, the program was reduced to eight tasks, with one task allocated per month rather than weekly. Precision cutting and precision suture placement were removed due to similarity to other program tasks and in response to specific feedback by 2015 participants. A summary of the program features is provided in Table 1.

3 112 Laparoscopic training at home TABLE 1 Comparison of box trainer program features in 2015 and Participants All gynaecology trainees (17) All gynaecology trainees (16) Intervention instrument eosim ProTrac 18 portable laparoscopic trainer, instruction manual and demonstration videos eosim ProTrac 18 portable laparoscopic trainer, instruction manual and demonstration videos Number of tasks 10 8 Task frequency Weekly Monthly Supervision of All received reminder s for each task All received reminder s for each task participants Half randomly allocated supervisor Half randomly allocated supervisor Outcomes Time to complete box trainer thread transfer Time to complete VR tubal ligation task Number of bleeding incidents in VR tubal ligation Time to complete box trainer thread transfer Time to complete VR tubal ligation and oophorectomy tasks Number of bleeding incidents in VR tubal ligation and oophorectomy Other measures Trainee logged practice time Trainee logged practice time VR, virtual reality Outcome measures Study outcomes included participant performance in laparoscopic tasks on the box trainer and a virtual reality (VR) simulator, as well as the amount of logged practice time. Laparoscopic performance was assessed with each participant completing a box trainer thread transfer task (video recorded by participants) and a VR bilateral tubal ligation at the beginning and end of each year s program. VR bilateral oophorectomy performance was also evaluated for participants in the 2016 group. Individual participant s performance was measured as the time to complete each task (in seconds), and the number of bleeding incidents for VR tasks (to evaluate error rate). Participants were instructed to record every training session, regardless of the duration, in their logbooks. Results at baseline were compared to post- program, as well as comparing performance outcomes and logged practice time between those in the supervisor group versus those in the no- supervisor group. Statistical analysis Only participants who completed both baseline and post- training assessments were included in the analysis. Paired and unpaired t- tests and Wilcoxon matched pairs signed- ranks tests or Mann Whitney U- tests were used for paired and unpaired, normally and non- normally distributed data respectively. Training outcomes were also evaluated for their relationship to logged practice (for those participants who recorded more or less than ten hours of training time) by examining the improvement in time taken to complete the thread transfer and VR tubal ligation tasks (as a percentage of the baseline time). A subgroup analysis was performed with trainees divided into early (unaccredited, years one and two), middle (years three and four) and late (years five and six) stages of training. Median improved times (as a percentage of baseline) and bleeding incidents were evaluated by Kruskal Wallis test across trainee levels. RESULTS 2015 study cohort In 2015, 17 trainees participated in the program. Participants included three non- trainee registrars, two first year, two second year, five third, two fourth, one fifth and two sixth year training registrars. The number of participants who completed both the baseline and post- training tasks is included in Table 2. Participants demonstrated significantly improved time to complete the VR tubal ligation following the program (median time at baseline 124 s vs 91 s post- program, P = 0.041) (see Table 2). The number of bleeding incidents was lower, but not significantly different post- training (Fig. 1). A smaller number of participants completed the thread transfer task post- program and a non- significant reduction in completion time was found study cohort In 2016, 16 trainees participated in the program, including three non- training registrars, two first year, two second year, three third year, two fourth year, two fifth year and two sixth year training registrars. Performance analysis revealed a significant reduction in the time taken to complete the VR tubal ligation and to complete the VR bilateral oophorectomy. A non- significant reduction in the number of bleeding incidents in both the tubal ligation and oophorectomy tasks was observed. There was no significant difference in thread transfer time. All participants Combining the two participant cohorts provided similar findings to the individual year analyses. Significant improvements were observed in the time and the median number of bleeding

4 E. Wilson et al. 113 TABLE 2 Performance outcomes: baseline and post- training program All participants Task Year Number of participants Baseline median (IQR) Post-training median (IQR) P-value Thread transfer (seconds) (33 48) 44 (27 46) (38 64) 52 (43 65) Combined (38 56) 46 (42 55) Tubal ligation time (seconds) (81 510) 71 (54 87) 0.021* (97 225) 91 (76 112) 0.041* Combined ( ) 79.5 (69 109) 0.002* Tubal ligation number of (6 8) 5 (3 7) bleeding incidents (5 10) 6 (4 8) Combined 20 7 ( ) 6 ( ) 0.010* Oophorectomy time ( ) 504 ( ) 0.025* (seconds) Oophorectomy number of ( ) 31.5 ( ) bleeding incidents Logged practice during ( ) program (time in minutes) ( ) Combined ( ) IQR, inter quartile range. *P < Only participants who completed both the baseline and post- training tasks were included in the analysis. The number of participants included in each analysis is given in the table. FIGURE 1 Performance outcomes for 2015 and 2016 groups. incidents in the tubal ligation task. No difference was found in the time to complete the thread transfer task. Supervisor allocation When training outcomes were analysed by supervisor allocation, no significant differences were found in performances between participants in the supervisor and no- supervisor groups (see Table 3). Logged practice Few participants completed the logbooks, but training times (for the duration of the program) ranged from 4.8 to 17.2 total training hours in the 2015 group (ten weeks) and 1.8 to 16.9 h in the 2016 group (eight months). When comparing outcomes with time spent practising, those participants who logged more than ten hours during the course of the programs appeared to demonstrate a higher relative improvement in time to complete the tubal ligation and thread transfer tasks (see Fig. 2), although the difference in median scores was not statistically significant. Trainee level Early trainees improved more in time to complete thread transfer task (median improvement 12.5%, interquartile range (IQR) 6 14%) compared to middle (median 17%, IQR 36 to

5 114 Laparoscopic training at home TABLE 3 Performance outcomes by supervisor allocation Supervisor median (IQR) No supervisor median (IQR) P-value Task Performance improvement (scores at baseline minus post- training) Thread transfer (seconds) 0 ( 4 to 3) 4 (0 4) Tubal Ligation time (seconds) 6 ( 14 to 57) 95 (33 201) Tubal ligation number of bleeding incidents 3 (1 4) 1 (0 4) Oophorectomy time (seconds) ( ) ( 0.5 to 1047) Oophorectomy number of bleeding incidents 6.5 ( 5 to 26) 6.5 (1 10.5) Training time (logged practice minutes) ( ) ( ) ( ) 784 ( ) Combined 290 ( ) 745 ( ) IQR, inter quartile range. Combined years 2015 and 2016 data. FIGURE 2 Relative improvement in performance times by practice amount. 2%) and late (median 7%, IQR 11 to 4%) staged trainees (P = 0.017). There was no significant difference across trainee level in other outcomes, with the majority of trainees improving post- training. DISCUSSION Improved performance in laparoscopic skills was observed following the implementation of this program using take- home box trainers. Trainees improved their time to complete a VR tubal ligation task (in 2015 and 2016 groups) and overall demonstrated a reduction in the number of bleeding incidents during the tubal ligation task. A reduction in time to complete a VR bilateral oophorectomy task was noted following the program in The program facilitated trainees to practise independently in their own time, without direct supervision or designated training time. Unfortunately not all participants completed the program requirements, reducing the data available for analysis and potentially signalling an underutilisation of the program. Previous research has demonstrated that laparoscopic training using both high and low realism trainers can produce improvements in laparoscopic performance. 10,11,26,27 Our study has demonstrated outcomes in line with the existing literature providing support for the use of box trainers. Despite the known benefits of simulation training, uptake of simulation in gynaecology training is low 12 and research into optimal methods of implementation is limited. 15 Our study has suggested a method of overcoming some of the known barriers to simulation (such as time and practical access to simulation), and the positive outcomes following training provide support for the program. This training may be of particular benefit to junior trainees (with relatively greater improvement in thread transfer time); however, the majority of trainees improved in performance regardless of seniority, indicating the benefit is not confined to those in early stages of training. A review of studies using take- home laparoscopic trainers internationally identified mixed results for performance benefits and reported underuse of off- site training. 28 Wide variation in logbook- recorded practice was identified in previous studies; from no practice to almost two hours per week Review authors noted a theme that trainees do not train as much as recommended. 28 Our study also identified suboptimal compliance with logbooks and performance tasks. Despite a longer program in 2016, total logged practice was similar, potentially signalling that a shorter course may be equally engaging (with potential benefits in terms of resource allocation). This must be balanced with the influences of shift work and other training priorities on the ability of trainees to complete tasks on a weekly schedule. Some participants failed to complete both the baseline and post- training tasks for a variety of reasons (such as leave or other commitments), reducing the available data for analysis. This failure of engagement with program requirements was seen in other take- home laparoscopic training studies. 22,23 Goal- directed proficiency- based

6 E. Wilson et al. 115 training and the provision for feedback are strategies suggested for take- home simulation training. 28 These features were incorporated in our program, yet engagement appeared low, highlighting the need for continued research into optimal curricula design. A detailed analysis of barriers to program engagement would be beneficial. In this study, supervisors were allocated to half of the participants to evaluate any potential impact on training. Research has suggested that supervision and feedback for training may be important components of simulation programs, 24,28,32 yet no significant differences in the performance outcomes of the two groups were identified in our study. The small number of participants may have precluded finding an effect from supervision. Alternatively, the feedback given may not have been able to alter performance as it was written only, a program feature intended to optimise the independence of the trainees. It is possible that participants were not additionally motivated to practise by such feedback or implied supervision. In considering the future role of take- home laparoscopic trainers, further research would be needed to inform the requirements for supervision. We were unable to assess the participants in real procedure performance due to the proportionally low number of surgeries performed of any one type in gynaecology at the study institution. There would not have been adequate opportunity for each participant to be evaluated in the same live procedure. It is recognised that each participant may have had different access to live procedures and other forms of laparoscopic training during the course of the program. Therefore performance improvements could be related to factors external to the simulation training. When comparing performance times at baseline to post- training, using VR tubal ligation as an example, the effect size was large and significant improvement was seen in both the 2015 and 2016 cohorts. This procedure is rarely performed in live surgery at this institution. Review of VR logs demonstrated that participants did not practise the tubal ligation task at other times throughout the programs. Also, the short duration of the 2015 cohort reduced the likely effect of standard hospital surgical training. Considering these factors, it is reasonable to conclude that the simulation training influenced the performance improvement seen. Additionally, we evaluated the tubal ligation performance scores for participants as a percentage improvement compared to the amount of logged training. Those who trained for more than ten hours during the programs appeared to trend toward greater improvements than those who trained less than ten hours. This supports the impression that the performance outcomes were related to the laparoscopic training. We did not intend to control for other forms of training, as it was not considered ethical or feasible to manipulate a trainee s individual exposure to surgical cases or training opportunities. We acknowledge that there will be fluctuations in any trainee s procedural training exposure and if simulation is considered an adjunct to traditional training rather than a replacement, it is not realistic to control the training received through the course of a study. Despite the acknowledged limitations, we were able to demonstrate an improvement in laparoscopic performance following training. Future programs may benefit from changes aimed at improving compliance with logging practise times and completing assessment tasks. A deeper understanding of the participants opinions of the program and factors influencing engagement would be beneficial. Making components of the program mandatory may improve the available data for analysis, yet has the potential to detract from the intentionally trainee- centred nature of the program. CONCLUSION In the two years of the take- home box trainer program, trainees demonstrated improvement in laparoscopic skills following the program. This program may serve as a means of overcoming existing barriers to accessing additional methods of surgical training in gynaecology, such as simulation. Further research would be beneficial to refine the essential components of curriculum design and increase trainee engagement with simulation training. ACKNOWLEDGEMENTS Dr Michael Beckmann and Dr Sarah Janssens are recipients of a Health Practitioner Research Fellowship from Mater Research. The authors acknowledge the assistance received from Donna Bonney (Mater Education) and Mater Education for providing the funding for the laparoscopic box trainers. REFERENCES 1. Obermair A, Tang A, Charters D et al. Survey of surgical skills of RANZCOG trainees. Aust N Z J Obstet Gynaecol 2009; 49: Driessen SRC, Janse JA, Schreuder HWR, Jansen FW. Proficiency for advanced laparoscopic procedures in gynecologic residency program: do all residents need to be trained? J Surg Educ 2015; 72: Smrtka MP, Gunatilake RP, Harris B et al. Increase in cesarean operative time following institution of the 80- hour workweek. J Grad Med Educ 2015; 7: Brolmann HA, Vervest HA, Heineman MJ. Declining trend in major gynaecological surgery in The Netherlands during Is there an impact on surgical skills and innovative ability? BJOG 2001; 108: Satava RM. Emerging trends that herald the future of surgical simulation. Surg Clin North Am 2010; 90: Ericsson KA. Deliberate practice and the acquisition and maintenance of expert performance in medicine and related domains. Acad Med 2004; 79: S70 S Reznick RK, MacRae H. Teaching surgical skills changes in the wind. N Engl J Med 2006; 355:

7 116 Laparoscopic training at home 8. Scott DJ, Bergen PC, Rege RV et al. Laparoscopic training on bench models: better and more cost effective than operating room experience? J Am Coll Surg 2000; 191: Dawe SR, Pena GN, Windsor JA et al. Systematic review of skills transfer after surgical simulation- based training. Br J Surg 2014; 101: Nagendran M, Toon CD, Davidson BR, Gurusamy KS. Laparoscopic surgical box model training for surgical trainees with no prior laparoscopic experience. Cochrane Database Syst Rev 2014; 1: Cd Gurusamy KS, Nagendran M, Toon CD, Davidson BR. Laparoscopic surgical box model training for surgical trainees with limited prior laparoscopic experience. Cochrane Database Syst Rev 2014; 3: Cd Wilson E, Janssens S, Hewett DG et al. Simulation training in obstetrics and gynaecology: what s happening on the frontline? Aust N Z J Obstet Gynaecol 2016; 56: Garber A, Posner G, El-Chaar D, Mitchell T. Letters to the Editor: simulation- based education in obstetrics and gynaecology training in Canada. J Obstet Gynaecol Can 2013; 35: Qayumi K, Pachev G, Zheng B et al. Status of simulation in health care education: an international survey. Adv Med Educ Pract 2014; 5: Stefanidis D, Sevdalis N, Paige J et al. Simulation in surgery: what s needed next? Ann Surg 2015; 261: Schreuder HW, van den Berg CB, Hazebroek EJ et al. Laparoscopic skills training using inexpensive box trainers: which exercises to choose when constructing a validated training course. BJOG 2011; 118: Ritter EM, Scott DJ. Design of a proficiency- based skills training curriculum for the fundamentals of laparoscopic surgery. Surg Innov 2007; 14: Hennessey IAM, Hewett P. Construct, concurrent, and content validity of the eosim laparoscopic simulator. J Laparoendosc Adv Surg Tech 2013; 23: Korndorffer JR, Arora S, Sevdalis N et al. The American College of Surgeons/Association of Program Directors in Surgery National Skills Curriculum: adoption rate, challenges and strategies for effective implementation into surgical residency programs. Surgery 2013; 154: Savoldelli GL, Naik VN, Hamstra SJ, Morgan PJ. Barriers to use of simulation- based education. Can J Anaesth 2005; 52: Brennan PM, Loan JJM, Hughes MA et al. Surgical training is undermined by inadequate provision of laparoscopic surgical simulators. Bull R Coll Surg Engl 2014; 96: Nicol LG, Walker KG, Cleland J On behalf of the Scottish Surgical Simulation Collaborative, including Royal College of Surgeons of Edinburgh, Royal College of Physicians and Surgeons of Glasgow, NHS Education for Scotland, et al. Incentivising practice with takehome laparoscopic simulators in two UK Core Surgical Training programmes. BMJ Simul Technol Enhanc Learn 2016; 2: Bennett J, Carter J, Fory C et al. Unsupervised use of laparoscopic box trainers in a cohort of UK core surgical trainees. J Surg Simul 2016; 3: Cole SJ, Mackenzie H, Ha J et al. Randomized controlled trial on the effect of coaching in simulated laparoscopic training. Surg Endosc 2014; 28: Snyder CW, Vandromme MJ, Tyra SL, Hawn MT. Proficiency- based laparoscopic and endoscopic training with virtual reality simulators: a comparison of proctored and independent approaches. J Surg Educ 2009; 66: Akdemir A, Ergenoğlu AM, Yeniel AÖ, Sendağ F. Conventional box model training improves laparoscopic skills during salpingectomy on LapSim: a randomized trial. J Turk Ger Gynecol Assoc 2013; 14: Nagendran M, Gurusamy KS, Aggarwal R et al. Virtual reality training for surgical trainees in laparoscopic surgery. Cochrane Database Syst Rev 2013, Issue 8. Art. No.: CD DOI: / CD pub Thinggaard E, Kleif J, Bjerrum F et al. Off- site training of laparoscopic skills, a scoping review using a thematic analysis. Surg Endosc 2016; 30: Zapf MA, Ujiki MB. Surgical resident evaluations of portable laparoscopic box trainers incorporated into a simulation- based minimally invasive surgery curriculum. Surg Innov 2014; 22: Griffin S, Kumar A, Burgess N, Donaldson P. Development of laparoscopic suturing skills: a prospective trial. J Endourol 2006; 20: Korndorffer JR Jr, Bellows CF, Tekian A et al. Effective home laparoscopic simulation training: a preliminary evaluation of an improved training paradigm. Am J Surg 2011; 203: Strandbygaard J, Bjerrum F, Maagaard M et al. Instructor feedback versus no instructor feedback on performance in a laparoscopic virtual reality simulator: a randomized trial. Ann Surg 2013; 257: