The effects of fatigue on robotic surgical skill training in Urology residents

Transcription

1 Thomas Jefferson University Jefferson Digital Commons Department of Urology Faculty Papers Department of Urology The effects of fatigue on robotic surgical skill training in Urology residents James R. Mark Thomas Jefferson University, Douglas C. Kelly Thomas Jefferson University, Edouard J. Trabulsi Thomas Jefferson University, Patrick J. Shenot Thomas Jefferson University, Costas D. Lallas Thomas Jefferson University, Let us know how access to this document benefits you Follow this and additional works at: Part of the Urology Commons Recommended Citation Mark, James R.; Kelly, Douglas C.; Trabulsi, Edouard J.; Shenot, Patrick J.; and Lallas, Costas D., "The effects of fatigue on robotic surgical skill training in Urology residents" (2014). Department of Urology Faculty Papers. Paper This Article is brought to you for free and open access by the Jefferson Digital Commons. The Jefferson Digital Commons is a service of Thomas Jefferson University's Center for Teaching and Learning (CTL). The Commons is a showcase for Jefferson books and journals, peer-reviewed scholarly publications, unique historical collections from the University archives, and teaching tools. The Jefferson Digital Commons allows researchers and interested readers anywhere in the world to learn about and keep up to date with Jefferson scholarship. This article has been accepted for inclusion in Department of Urology Faculty Papers by an authorized administrator of the Jefferson Digital Commons. For more information, please contact: JeffersonDigitalCommons@jefferson.edu.

2 The Effects of Fatigue on Robotic Surgical Skill Training in Urology Residents James R Mark, MD, Douglas C Kelly, MD, Edouard J Trabulsi, MD, FACS, Patrick J Shenot, MD, and Costas D Lallas, MD, FACS Department of Urology, Thomas Jefferson University Hospital, Philadelphia, PA Keywords: surgical training; fatigue; robotic surgery; virtual reality simulation Corresponding author: Costas D Lallas, MD, FACS Associate Professor of Urology Director of Robotic Surgery Thomas Jefferson University Hospital 1025 Walnut Street, suite 1100 Philadelphia, PA office fax costas.lallas@jefferson.edu

3 Abbreviations VR virtual reality dvss davinci Skills Simulator ESS Epworth Sleepiness Scale

4 Abstract Purpose This study reports on the effect of fatigue on Urology residents using the davinci surgical skills simulator (dvss). Materials and Methods Seven Urology residents performed a series of selected exercises on the dvss while pre call and post call.prior to dvss performance a survey of subjective fatigue was taken and residents were tested with the Epworth Sleepiness Scale (ESS). Using the metrics available in the dvss software, the performance of each resident was evaluated. Results The urology residents slept an average of 4.07 hrs (range hrs) while on call compated to an average of 5.43 hrs while not on call (range 3 7hrs, p= 0.08). Post call residents were significantly more likely to be identified as fatigued by the Epworth Sleepiness Score than pre call residents (p=0.01). Significant differences were observed in fatigued residents performing the exercises Tubes and Match Board 2 (p = 0.05, 0.02). Additionally there were significant differences in the total number of critical errors during the training session (9.29 vs 3.14, p = 0.04). Conclusions Fatigue in post call Urology residents leads to poorer performance on the dvss simulator. The dvss may become a useful instrument in the education of fatigued residents and a tool to identify fatigue in trainees.

5 Introduction In July, 2003 all ACGME accredited residency programs adopted a policy to restrict resident clinical activities to 80hours per week with further restrictions applied to interns July, These rules were enacted in an effort to increase patient safety by reducing errors made by fatigued resident physicians. Sleep deprivation has been shown to cause impairment similar to alcohol intoxication. 2 Taffinder et al. have used surgical simulators to correlate a decline in surgical proficiency with sleep deprivation. 3 This observation has been supported with subsequent studies post duty hours restrictions showing deficiencies in psychomotor and cognitive skills in tired surgical trainees. 4, 5, 6, 7 However, like with many simulation-based training regimens, we are somewhat behind when compared to other fields. The effects of fatigue on performance has long been studied and established, and in fact in aviation, simulation is being used to mimic fatigue and help develop strategies to mitigate its effects on performance. 8,9 Urology residency is unique to other surgical training programs in that successful completion requires competency in open, endoscopic, laparoscopic, and now robotic assisted laparoscopic surgery. To assist in the acquisition of these skills, virtual reality (VR) simulators have been developed. One such simulator is the davinci Skills Simulator (dvss) produced by Intuitive Surgical (Sunnyvale, CA). This is the first robotic surgical simulator available to allow surgical skills training in a virtual environment using an actual davinci Si surgeon s console. The dvss is a backpack mounted to a davinci Si console that powers software produced by Mimic Technologies (Seattle, WA). 10 Surgeon performance is graded using MScore, a program that compares metric data to the performance of over 100 expert surgeons. This system has recently been shown by two independent researchers to show face, content, and construct validity. 11,12

6 The effect of fatigue on Urology residents performing robotic procedures has not been examined. The purpose of this study is to use the dvss to test the effect of fatigue on Urology residents surgical skills required for performing robotic surgery. We hypothesize that residents will show 4, 5, deficiencies as previously reported in general surgery residents using laparoscopic simulators. 6, 7 Methods Seven Urology residents post graduate years 2 through 6 performed a total of 14 training sessions on the dvss in this prospective study. For each resident one of those sessions was intentionally performed following a night on call and the other off call. Before each training session the resident filled out a questionnaire rating self-perceived fatigue from 1 to 10 (1 = no fatigue), detailing the number of hours slept the previous night, number of interruptions in sleep, and if the resident was post call. Residents were asked to not spend any time on the simulator between sessions, so as not to influence the results and instead capture the true effect, if any, of fatigue on performance; thus the residents served as their own internal control. Finally, 3 of the residents did not perform the exercises in sequence of pre-call to post-call, but instead performed their post-call exercised first; this helped to eliminate any practice-related improvement on the simulator from biasing the results. Additionally, each resident was tested for fatigue using the Epworth Sleepiness Scale (ESS), a previously validated measure of daytime sleepiness. (Appendix) 13,14

7 Each training session included completion of 5 exercises (Figure 1) representing all major skill sets tested by the dvss; the dvss software contains 33 skills tests divided into 5 skill set categories. Prior to each exercise, the resident viewed an instructional video detailing the skills to be performed. MScore was used to calculate a total score based on the following metrics: Time to complete exercise, economy of motion, instrument collisions, excessive instrument force, instruments out of view, and master workspace range. Additional, task-specific metrics such as misapplied energy time, needle drops, and missed targets were also used. A score of less than 60th percentile on any individual metric within a training exercise resulted in a critical error, which was added to the individual s performance data. An ESS score greater than 10 is indicative of greater than an average amount of daytime sleepiness and was therefore used as the cutoff value to create fatigued and non-fatigued groups. 13,14 Pre and post call status was not used as not all post call residents were fatigued and not all pre call residents were rested. All continuous variables were explored for normality using the Shapiro-Wilk test. As about half had non-normal distributions, Wilcoxon two-sample tests were used to test for comparisons and medians, minimums and maximums were reported as descriptive statistics. Using the non-parametric test also allow for testing of the critical error scores, which as counts would not be adequately compared using a t-test. Categorical comparisons used Fisher's exact test, due to the small sample size. Results The Urology residents slept a median of 4 hours (range hours) while on call and a median of 5 hours while not on call (range 3 7 hours, p= ). Median interruptions of sleep on call

8 were 2 vs. 1 (p=0.0685) while not on call. Post call residents were significantly more likely to be identified as fatigued by the Epworth Sleepiness Scale than pre call residents (p=0.0479). Median scores for each exercise are presented in Table 2. Significant differences were observed in fatigued residents performing the exercises Tubes (Figure. 1c) and Match Board 2 (Figure 1d) (p = , ). Additionally there were significant differences in the total number of critical errors during the training session (2 versus 6, p = ). Analysis of each metric recorded for individual exercises is shown in Table 3. The one single component that shows the most difference is time-to-completion, which is significant or approaching significance on nearly all domains. Discussion Resident fatigue has been a prominent topic in American surgical education since the enactment of duty hour restrictions by the ACGME in July The American College of Surgeons (ACS) has voiced concern that restricted hours may negatively impact resident training in surgery. Many of the objections raised by the ACS extend from the lack of evidence that resident fatigue has led to unsafe conditions for surgical patients and that the initial studies prompting the change focus primarily on medical residents. 15 There is additional concern that the work-hour restrictions may lead to inadequate training in Urology. In a poll of Urology Program Directors, 43% of those who responded disagreed that residents could be properly trained in the new time frame imposed by the ACGME. 16 Finally, although recent data report that for surgeons who may be sleep deprived, surgical outcomes are not necessarily compromised, it is important to

9 distinguish that these studies focus on experienced surgeons, whose maneuvers are well practiced and automatic. 17,18 With trainees, however, these impacts will be significantly magnified because of the inexperience of the subject to the operations. In lieu of these concerns it is surprising that very little research has been done on the impact of fatigue on residents in Urology training programs. To our knowledge, this is the first study to examine not only the performance of fatigued Urology residents, but also the effect of fatigue on the skill sets used for robotic assisted surgery. We found that Urology residents with an Epworth sleepiness scale score greater than 10 demonstrated significantly poorer performance on the dvss exercises Tubes and Match Board 2. In addition, a significantly greater number of critical errors were recorded for fatigued residents. The dvss exercises performed were developed to train the user in different skills required to perform robotic surgery with the davinci. Ring Walk 2 (Figure 1a) is and exercise that requires the operator to move a ring over a vessel while intermittently adjusting the camera to focus on appearing targets. This exercise showed no significant difference between fatigued and nonfatigued residents. Our previous experience validating the dvss found no significant difference in basic camera targeting exercises between novice, intermediate, and experienced groups. 11 It may therefore be too simple a task to demonstrate variation with fatigue as none of the metrics measured by the dvss for this exercise showed significant change, however, examination of the individual metrics recorded for this exercise demonstrated a significant increase in time to completion for fatigued residents (106.6 vs sec, p= ).

10 Time to completion was also significantly increased in Energy Dissection 2 (Figure 1b). This is an exercise where small vessels are cauterized by bipolar Maryland graspers and then cut with monopolar scissors. The standard metrics are recorded by MScore as are misapplied energy time and broken vessels. Blood loss is recorded and was shown to be significantly higher in fatigued residents (201 vs. 84, p = ). Thread the rings (Figure 1e) is an exercise where a needle is sequentially passed through a series of rings in different orientations to help develop muscle memory for needle driving. MScore records needle drops in addition to the standard metrics. This exercise failed to demonstrate a significant difference in any metric. Significant differences in overall score were noted in both Tubes (Figure 1c) and Match board 2 (Figure 1d). Tubes is an exercise used to practice the movements required to perform an anastomosis. It is likely that the difficulty experienced by fatigued residents on tubes is due to a decrease in the fine motor skills required for this challenging exercise. 6 Studies on the learning curve for robotic assisted laparoscopic radical prostatectomy show the vesicourethral anastomosis to be one of the more difficult steps for the novice robotic surgeon in vivo. 19 Recreation of this skill set on a virtual platform with the Tubes exercise is likewise challenging and is impacted by fatigue. Match board 2 (Figure 1d) is a unique exercise that combines many skills. The operator must first lift a hinged lid to expose squares labeled with letters and numbers. Scattered around the board are blocks shaped into alphanumeric characters that are grasped using robotic needle

11 drivers and placed on their corresponding square. The distance of the block from the board requires extensive use of the clutch to limit the master workspace range. Analysis of the metrics recorded by MScore for this exercise show a significant difference in the time to complete this exercise (138.1 vs sec, p= ) by fatigued residents. These results show that fatigued Urology residents have decreased fine motor skills and reaction time while using a robotic simulator. Declining fine motor skill is demonstrated by the significant difference in overall score on Tubes. Additionally, decreased reaction time is evident by the poor ability to recognize and control bleeding on Energy dissection 2 as well as an increased time taken to recognize alphanumeric characters on Match board 2. Even though not all differences were statistically significant in this study, it is interesting to note that every exercise performed had a lower median score when performed by fatigued residents. We do acknowledge several limitations to our study. Our residents surprisingly slept just one hour longer when not on call as compared to on call. We believe that this is representative of a surgical residency; although one may not be trolling the emergency department and fielding calls, there is work that nonetheless still has to be done, both medicine and otherwise related. Also, when looking at the quality of the sleep between the two groups, those on call were expectedly significantly more fatigued as identified by the ESS. As we included only Urology trainees we have a small sample size of 7 residents. Details on the individual residents experience with the dvss or with experience performing or assisting with robotic procedures was not accounted for and they were not blinded. Additional variables such as natural circadian shifts in alertness were not controlled for as exercises were performed at differing times of day

12 due to resident availability. Future work on this topic should attempt to control for these variables. We also realize that there can be an array of interruptions that may impact a night on call. Although we did not qualify the nature of these breaks in sleep, we did believe that an absolute number of interruptions would give a sufficient idea of how the sleep cycle had been disrupted. Fatigue in surgery is a reality that residents will face upon completion of their urology training when work hour restrictions do not apply. It is therefore crucial that surgical educators identify skills that may be impaired by fatigue and attempt to improve performance by tired residents. Naughton et al. have demonstrated that virtual reality training in post call residents leads to proficiency in endovascular surgical techniques. While acquisition of skills took longer in fatigued residents than in non-fatigued residents, performance for the two groups eventually equilibrated. 20 Applying this knowledge to our study implicates the dvss as a tool for improving performance in fatigued Urology residents. Further research should be done to verify this claim. Conclusions Our study demonstrates that fatigued Urology residents show a significant decline in fine motor skills and reaction time using the dvss. There is definite potential for such a system to measure fatigue and performance in trainees in order to optimize training and reduce errors in the operating room and beyond. These exercises can be used for further research on robotic surgical skill acquisition in fatigued Urology residents.

13 Acknowledgements Statistics performed by Jocelyn A. Andrel, MSPH, Division of Biostatistics, Thomas Jefferson Medical College, Philadelphia, PA. Legends to Figures Figure 1: davinci Skills Simulator (dvss) skills tests Figure 1a: Ring Walk 2 Figure 1b: Energy Dissection 2 Figure 1c: Tubes Figure 1d: Match Board 2 Figure 1e: Thread the Rings Appendix: Epworth Sleepiness Scale (ESS) James R Mark, Douglas C Kelly, Patrick J Shenot, Edouard J Trabulsi and Costas D Lallas declare that they have no conflicts of interest.

14 References 1. ACGME s Common Program Requirements. Available at: Accessed on 10/10/ Dawson D, Reid K. Fatigue, alcohol, and performance impairment. Nature 1997;388: Taffinder NJ, McManus IC, Gul Y, et al. Effect of sleep deprivation on surgeons dexterity on laparoscopy simulator. Lancet 1998;352: Kahol K, Leyba MJ, Deka M, et al. Effect of fatigue on psychomotor and cognitive skills. American journal of surgery 2008;195: Leff D, Aggarwal R, Rana M, et al. Laparoscopic skills suffer on the first shift of sequential night shifts: Program directors beware and residents prepare. Annals of Surgery 2008;247(3): Eastridge BJ, Hamilton EC, O Keefe GE, et al. Effect of sleep deprivation on the performance of simulated laparoscopic surgical skill. American Journal of Surgery 2003:186; Gerdes J, Kahol K, Smith M, et al. Jack Barney award: The effect of fatigue on cognitive and psychomotor skills of trauma residents and attending surgeons. American Journal of Surgery 2008:196;

15 8. Fatigue Risk Management in Aviation Maintenance: Current Best Practices and Potential Future Countermeasures, FAA, Washington, DC 20591, June Simulation of Biomathematical Models to support Fatigue Risk Management Systems (FRMS) in Training and Simulation. September Accessed 3/4/ Lallas CD, Davis JW et al: Robotic surgery training with commercially available simulation systems in 2011: A current review and practice pattern survey from the Society of Urologic Robotic Surgeons. J Endourol Mar;26(3): Kelly, DC, Margules AC, Kundavaram CR, et al. Face, content and construct validation of the da Vinci skills simulator. Urology 2012; 79(5): Hung AJ, Zehnder P, Patil MB, et al. Face, content, and construct validity of a novel robotic surgery simulator. J Urol 2011;186: Johns MW. A new method for measuring daytime sleepiness: the Epworth sleepiness scale. Sleep 1991;14: Johns MW. Reliability and factor analysis of the Epworth sleepiness scale. Sleep 1992;15: Britt LD, Sachdeva AK, Healy GB, et al. Resident duty hours in surgery for ensuring patient safety, providing optimum resident education and training, and promoting resident well-being: A response from the American College of Surgeons to the Report of the Institute of Medicine Resident Duty Hours: Enhancing Sleep, Supervision, and Safety. Surgery 2009:146;

16 16. Joyner BD, Siedel K, Stoll D, et al. Report of the national survey of urology program directors: Attitudes and actions regarding the accreditation council for graduate medical education regulations. J Urol 2005:174; Rothschild JM, Keohane CA, Rogers S, et al. Risks of complications by attending physicians after performing nighttime procedures. JAMA Oct 14;302(14): Vinden C, Nash DM, Rangrej J, et al. Complications of daytime elective laparoscopic cholecystectomies performed by surgeons who operated the night before. JAMA Nov 6;310(17): Dev H, Sharma NL, Dawson SN, et al. Detailed analysis of operating time learning curves in robotic prostatectomy by a novice surgeon [published online ahead of print Oct 28, 2011]. BJU International. DOI: /j X Naughton PA, Aggarwal R, Wang TT, et al. Skills training after night shift work enables acquisition of endovascular technical skills on a virtual reality simulator. Journal of Vascular Surgery 2011;53: