ORIGINAL ARTICLE. Objective Evaluation of a Laparoscopic Surgical Skill Program for Residents and Senior Surgeons

ORIGINAL ARTICLE Objective Evaluation of a Laparoscopic Surgical Skill Program for Residents and Senior Surgeons James C. Rosser, Jr, MD; Ludie E. Rosser, BSc; Raghu S. Savalgi, MD, PhD(Surg), FRCS Background: Laparoscopic surgery adapts poorly to apprenticeship models for general surgical training. Standardized skill acquisition and validation programs, targeted performance goals, and a supervised, enforced, skill-based curriculum that readily can be shared between trainee and instructor must replace the observation and incremental skill-acquisition model used in an open surgical environment. The Yale Laparoscopic Skills and Suturing Program was used to develop a data bank for objective evaluation of dexterity and suturing skills for laparoscopic surgical training. The current study compares trainee and senior surgeon performance in this standardized training program. Objective: To compare objectively evaluated laparoscopic surgical skills and suturing capability of senior surgeons and of residents after they have completed the same standardized training regimen. Methods: Two hundred ninety-one trained surgeons performed 873 standardized laparoscopic dexterity drills and 291 intracorporeal suturing exercises in the Yale Laparoscopic Skills and Suturing Program. Their performance was supervised by an instructor who recorded performance and timing of the tasks in a 2 1 2-day program. Ninety-nine residents performed the same drills and exercises the same number of times and followed the same technique for intracorporeal suturing. Percentile graphs were prepared for each type of drill and suturing exercise to allow comparison of levels of achievement among different training groups. Results: The performance of the residents was the same as that of trained surgeons for the rope pass drill and the suturing exercise. Residents in comparison with trained surgeons performed the triangle transfer drill faster and the new cup drop drill and old cup drop drill more slowly. There was no significant difference in performance between male and female residents. Conclusion: Basic skills relevant to laparoscopic performance can be acquired with a high level of competence in a brief course unrelated to prior surgical experience, sex, or age. Arch Surg. 1998;133:657-661 From the Department of Surgery, Yale University School of Medicine, New Haven, Conn. THIS ARTICLE reports on the Yale Laparoscopic Skills and Suturing Program, a training method 1-3 for objective evaluation of laparoscopic surgical skills including intracorporeal suturing. Since the introduction of laparoscopic general surgery, many courses to retrain established surgeons have emerged. However, objective parameters are not commonly used to assess the outcome of courses. 4-6 The safe and efficient inclusion of laparoscopic surgery as an integral part of general surgery demands dissemination of laparoscopic surgical skills among trained surgeons and residents with a common platform. An effective standardized skill-acquisition course with objective evaluations can facilitate skill transfer for the retraining of senior surgeons and establishing initial surgical training in residents. 7-9 Our aim was to evaluate the effectiveness of a method of teaching basic laparoscopic skills and suturing to trained surgeons and residents. A database with percentile graphs was created to allow valid comparison of the skills of the participants (Figure 1). RESULTS The total time in seconds required by 291 trained surgeons (268 male and 23 female) and 99 residents (78 male and 21 female) to perform 1 drills and 1 suturing exercises of each type is shown in Table 1. The resident group included 61 general surgical residents and 19 gynecology residents across the spectrum from introduction to chief residency, and 19 preresidency participants. The senior surgeons were older than the residents (mean ± SEM, 44±.55 years vs 31±1.15 years, 657

PARTICIPANTS All 291 trained surgeons were board eligible or board certified or had equivalent qualifications and experience. All were previously certified to perform laparoscopic surgery in their hospitals. They attended a standardized training program for 2 1 2 days, as described previously. 1-3 All the trained surgeon participants performed 3 types of standardized drills (rope pass drill, triangle transfer drill, and cup drop drill) 1 times each. They were also trained in a standardized intracorporeal suturing technique. The instructor-trainee ratio was 1:4. Data were collected by the instructors, who ensured the competency and timing of each drill and suturing technique. All exercises, including standardized intracorporeal anastomosis (8 cm long, 2 layers), were performed within the a laparoscopic trainer (Surgi- Trainer, US Surgical Corporation, Norwalk, Conn) using a Sony monitor (Trinitron-model PVM-1943MD, Sony, Tokyo, Japan) and a, 1-mm laparoscope (Stryker Endoscopy, Santa Clara, Calif) and a 3-chip medical video camera (Stryker model 777). The participants performed 3 types of drills to improve their dexterity, depth perception, instrument targeting accuracy, visual and spatial abilities, and hand-eye coordination. Use of the nondominant hand was emphasized. These are generalized skill requirements, applicable not only to laparoscopic surgery but also to other types of endoscopic surgical techniques. The drills are subset tasks relevant to increasing efficiency in laparoscopic suturing. The steps involved in intracorporeal suturing are described in detail elsewhere. 1,6 A detailed suturing algorithm was developed and strictly followed. The process was broken into 3 phases: needle positioning, tissue penetration, and knotting. Only 1 practice drill was allowed, and thereafter all the exercises were timed in a standardized fashion. The participants performed the drills before the suturing exercises on each day of the 2 1 2-day course. The data collection for trained surgeons began in 1991 and ended in 1994. The same course was conducted for residents at Yale University School of Medicine over a 3-year period (from 1994 to 1996). The residents attended the course in batches of 15 to 2 participants. The courses were conducted during a weekend and were an essential requirement of the residency training program. At the end of the course, each resident was given an analysis indicating the resident s percentile performance compared with that of the data bank of trained surgeons. The time required to perform each dexterity drill or suturing exercise was documented in seconds. The 291 trained surgeons who performed 1 of each drill or suturing exercise were included in the final analysis. The total time required to perform the first 1 drills and suturing exercises was calculated for each participant and used for comparison with all the calculations. This method yielded 1 value in seconds per participant per drill or suturing exercise. Cumulative percentile graphs were prepared for all the drills and suturing exercises. 1,11 The comparisons of residents with trained surgeons and of male with female participants were made with unpaired, 2-tailed t tests. Seconds 5 Cup Drop (New) Cup Drop (Old) Rope Pass Drill Triangle Transfer Drill 1 2 3 4 5 6 7 8 9 1 Percentiles Figure 1. Reference percentile graphs prepared from the data bank of trained surgeons. P.1), as expected. The container used to perform the cup drop drill was changed to a porcelain cup with an aperture, as it was difficult to obtain the former type of cup.this alteration, however, increased the difficulty in performing the cup drop drill, thus requiring a longer time to complete it. Separate analysis was, therefore, performed for both old and new cup drop drills. The drill with the larger sample size was evaluated unless the amalgamation of data did not influence the outcome of the analysis. A comparison of time requirements for male and female residents and for male and female trained surgeons is shown in Table 2 and Table 3, respectively. Each resident s performance in drills and suturing exercises was compared with the percentile graphs of trained surgeons, and an accurate percentile rank for each resident was thus determied. The time required to perform 1 drills and suturing exercises for the residents as a group and for the trained surgeons are compared in Table 1. Data of 3 categories of residents are given separately, so that the performance of these subgroups can be appreciated on their own merit. Data of all the residents and data of their subgroups were compared separately with the performance of the trained surgeons; P values in Table 1 refer to these comparisons. There was no significant difference in the time requirement of residents and trained surgeons in performing the rope pass drill and suturing exercises. Furthermore, residents took significantly less time to perform the triangle transfer drill. They required more time than the trained surgeons, however, to perform the old cup drop drill, new cup drop drill, and an 8-cm, 2- layer anastamosis. Data given in Table 1 for subgroups of residents are informative for evaluating subgroup behavior in comparison with that of all of the residents as a whole. The time requirements of male and female residents and male and female trained surgeons are shown in Tables 2 and 3, respectively. No significant difference was noted in the performance of male and female residents. However, the trained female surgeons required 658

Table 1., Trained Surgeons vs Residents* Trained Surgeons (n = 291) All Residents (n = 99) General Surgery Residents (n = 61) Resident Subgroups Gynecology Residents (n = 19) Preresidency Participants (n = 19) Time P Time P Time P Time P Rope pass drill 69 ± 1 676 ± 18.1 687 ± 23.1 63 ± 44.1 681 ± 42.1 Triangle transfer drill 445 ± 9 385 ± 16.5 389 ± 2.5 346 ± 37.5 411 ± 38.1 Old cup drop drill 89 ± 23 (n = 145) 15 ± 43 (n = 43).5.................. New cup drop drill 1233 ± 3 (n = 146) 1371 ± 48 (n = 56).5.................. Suturing exercises 1839 ± 38 186 ± 67.1 1673 ± 84.1 1555 ± 156.1 2466 ± 19.1 *Time in seconds, mean ± SEM, required to complete 1 drills and suturing exercises by trained surgeons, residents, and subgroups of residents. Completion times of residents as a group and of 3 subgroups of residents are compared with those of trained surgeons by P values derived with unpaired, 2-tailed t tests. In the old cup drop drill and the new cup drop drill, the numbers of participants are listed separately from the numbers in other types of drills and exercises. Ellipses indicate category of data not applicable. Table 2., Male vs Female Residents* Male (n = 78) Female (n = 21) Rope pass drill 67 ± 19 699 ± 37 Triangle transfer drill 38 ± 16 42 ± 31 New cup drop drill 1358 ± 63 1423 ± 124 (n = 46) (n = 12) Suturing exercises 1815 ± 74 1773 ± 2 *Time in seconds, mean ± SEM, required to complete 1 drills and suturing exercises by male and female residents. P values were derived with unpaired, 2-tailed t tests. All P values are.1. In the new cup drop drill, the numbers of participants are listed separately from the numbers in other drills and exercises., s 5 15 25 Cup Drop Drill, s Figure 2. Correlation between time required to perform 1 rope pass drills and 1 suturing exercises (r=.51, P.1). Table 3., Male vs Female Trained Surgeons* Male (n = 268) Female (n = 23) P Rope pass drill 683 ± 11 779 ± 38.5 Triangle transfer drill 443 ± 1 478 ± 34.1 Old cup drop drill 1619 ± 42 1848 ± 122.1 (n = 187) (n = 23) Suturing exercises 182 ± 4 2156 ± 14.5 *Time in seconds, mean ± SEM, required to complete 1 drills and suturing exercises by male and female trained surgeons. P values are derived with unpaired, 2-tailed t tests. In the old cup drop drill, the numbers of participants are listed separately from the numbers in other drills and exercises. more time than trained males to perform the rope pass drill and suturing exercises. No significant difference was found between the 2 groups in the time required to perform the old cup drop drill and the triangle transfer drill. The times required to perform 1 drills and 1 suturing exercises were tested for correlation in the total study population of 392 that included all the trained surgeons and all the residents. A significant correlation was noted between the time required for the suturing exercises and the time recorded in all 3 drills (rope pass drill,, s 3 6 9 12 15 18 Rope Pass Drill, s Figure 3. Correlation between time required to perform 1 triangle transfer drills and 1 suturing exercises (r=.53, P.1). n=352, r=.51, P.1, Figure 2; triangle transfer drill, n=354, r=.53, P.1, Figure 3; cup drop drill, n=354, r=.51, P.1, Figure 4). The residents were younger than the trained surgeons (mean ± SEM, 3.79±1.15 years vs 43.94±.55 years, P.1), as expected. The age of all the participants (41.46±.58 years) including trained surgeons and residents had very low correlation with the time required to perform 1 drills and 1 suturing exercises (rope pass drill, n=274, r=.17, P.5; triangle transfer drill, n=277, r=.23, P.5; cup drop drill, n=278, r=.7, P.1; suturing exercises, n=244, r=.27, P.1). 659

, s 1 3 5 7 9 11 Triangle Transfer Drill, s Figure 4. Correlation between time required to perform 1 new or old cup drop drills and 1 suturing exercises (r=.51, P.1). COMMENT Traditional educational strategies in surgery need to be altered to cope with recent developments in minimally invasive surgery. Current training systems are mainly dependent on subjective evaluation of trainee surgeons by the surgical preceptors, a loosely organized curriculum, and cognitive testing through multiple-choice questions. Very few training programs use objective tests of dexterity and surgical skills in their usual surgical training and evaluation methods. This is probably justified because the relationship between surgical skills and manual dexterity has been questioned. For example, some investigators did not find any difference in manual dexterity between physicians and surgeons. 12 In addition, others 13 did not find any correlation between surgical clinical proficiency and psychomotor skills. These studies, however, were not conducted in a laparoscopic environment and results cannot be extrapolated to laparoscopic surgery. It is agreed that performance in an objective evaluation should not be taken as the sole factor in judging the clinical or technical competency of a surgeon. Cognitive tests and objective evaluation of dexterity and psychomotor skills assess different characteristics of a surgeon. While tests of cognitive knowledge represent a valid measure of the retention of factual material, they cannot be used as the sole basis for assessing clinical competence in a residency program. 14 Objective evaluations should be used in conjunction with subjective and cognitive tests and should not replace them. Correlation can be observed between the teaching of gross motor skill development by physical educators and the teaching of fine manual motor skills associated with surgical skill development. 15 The need for a reliable method of skill acquisition was documented by a survey of 152 program directors in the United States, 16 5% of whom felt that there was a need for improving the establishment of technical skill. No single method has been accepted as the best by any group, but many are being explored. 17 The concept of skill labs is widely accepted, 18 but curriculum content must dictate the effectiveness of these labs to affect skill transference. Minimal-access surgery demands high-quality assurance standards in order for patients to reap maximum benefits. 19 Whatever transpires from the emerging technology, we must not overlook surgical principles governing operative intervention, which apply regardless of the approach and instrumentation. High performance standards with widespread availability cannot become a reality without the estalishment of skills. Some programs have tried to introduce skill development more systematically than others. 19,2 They did not, however, use objective parameters of laparoscopic skill, but only recorded an increase in the total number of laparoscopic cholecystectomies performed by the trainees in their hospitals. Coppola and Merrell 21 discussed outcome reference points in their study on the effect of a laparoscopic skill-development regimen on a resident training program. After residents participated in this program, the academic level of the residents able to perform a laparoscopic cholecystectomy significantly decreased without extended operative times or increased complication rates. The objective evaluation of dexterity and skill can only become a reality if a large database is established. There are no such data banks in conventional surgery, but this database for laparoscopic surgery could prove very helpful in the development of a training system for laparoscopic surgery. A database provides a way for participants to know how successful their efforts are in attaining a level of competence. It gives them a performance reference point the skill levels of trained surgeons to try to achieve and to pass. It also helps to establish effective training countermeasures for participants who have difficulty. The relationship between dexterity drills and suturing techniques that was noted in our initial reports 1-3 was also found in the current study (Figures 2-4). This strongly supports correlation between dexterity skills and a meaningful surgical task. It has been a commonly held assumption that younger surgeons have a natural advantage in the development of laparoscopic skills. Some authors reported that age influenced pure motor skills but neither age nor pure motor skills are necessarily important for operative skills. Age did not appear to play a dominant role in the outcome of our courses. All the participants who attended our courses, regardless of age, sex, and previous training, learned intracorporeal suturing and performed anastamosis. There was no participant whose performance did not improve. The correlation between age and the time required to perform the dexterity drills and intracorporeal suturing was very low. All the residents as a group took marginally longer to complete an anastamosis, although their suturing time was not significantly different from that of the trained surgeons. This difference may be due to a lack of experience in performing anastamosis rather than in suturing. Residents required more time than the trained surgeons to perform the cup drop drill, which requires considerable depth-perception compensation to the 2-dimensional environment. However, residents performed significantly better than the trained surgeons in the triangle transfer drill. It is possible that age and experience might influence some types of dexterity drills, but overall they do not seem to play a dominant role. Laparoscopic intracorporeal suturing is probably the most difficult exercise to master in the minimally inva- 66

sive environment. The drills in this program were associated with mastery of the suturing process. Both the residents and the trained surgeons reached a comparable efficiency with the same educational program. It is notable that the residents were effectively taught intracorporeal suturing in the same training program as were the attending surgeons. The residents did not have the same experience as senior surgeons in constructing an intestinal anastomosis in the open environment. Although their anastomosis completion times were longer than those of the trained surgeons, they developed a foundation on which to competently build (Table 1). There was no significant difference (P.1), however, between the intracorporeal suturing times of the residents as a group vs the trained surgeons. The residents performance is all the more noteworthy considering that 7% of board eligible, trained surgeons, before taking our course, could not complete a knot in 5 minutes. 1 Some authors reported that female residents scored better in cognitive tasks but scored less well in visualmotor tasks in comparison with male residents. In our study there does not appear to be a dominant influence of sex on laparoscopic skill acquisition. No difference was noted in the performance of male and female residents in performing drills and suturing exercises. The finding that trained females took a longer time to complete suturing exercises and rope pass drills was only marginally significant (P.5). Hayward et al 22 compared the abilities of male and female residents in 6 areas: ethics, judgment, technical skills, knowledge, interpersonal skills, and work habits. A 5-point scale was used for evaluation. They did not find any difference between male and female residents. This finding was also consistent with our current study. CONCLUSIONS The Yale Laparoscopic Skills and Suturing Program is effective in establishing laparoscopic and suturing skills in surgeons and residents regardless of age or sex in a short time, resulting in greater teaching efficiency. Because the database generated by this study is applicable to a multilevel spectrum of participants, comparison of skill levels of residents against the percentile graphs derived from the performance of trained surgeons is possible. Participants who finish this standardized curriculum can compare their performance with that of trained surgeons, giving them an initiative to practice and improve. Reprints: James C. Rosser, MD, Department of Surgery, Yale University School of Medicine, 4 Temple St, Suite 3A, New Haven, CT 651. REFERENCES 1. Rosser JC, Rosser LE, Savalgi RS. Skill acquisition and assessment for laparoscopic surgery. Arch Surg. 1997;132:2-24. 2. Rosser JC, Rosser LE, Savalgi RS. Objective evaluation in laparoscopic surgical training and credentialing. Surg Endosc. 1996;1:257. 3. Rosser JC, Rosser LE, Savalgi RS. Objective comparison of caparoscopic surgical skills in trained and trainee surgeons. In: Program and Abstracts, Fourth International Congress of the European Association for Endoscopic Surgery. Trondheim, Norway: Springer-Verlag; 1996:57. Abstract. 4. Rosser JC, Rosser LE, Savalgi RS. Computer-based training in laparoscopic surgery. Surg Endosc. 1996;1:257. 5. Rosser JC, Rosser LE, Savalgi RS. Computer based training in laparoscopic surgery. Surg Endosc. 1996;1:565. 6. Rosser JC. The Art of Laparoscopic Suturing [interactive training module on CD- ROM]. Oxford, England: Radcliffe Medical Press; 1995. 7. Rosser JC, Rosser LE, Savalgi RS. Telementoring: a practical option in surgical training. Surg Endosc. 1997;11:852-858. 8. Rosser JC, Rosser LE, Savalgi RS. Incorporation of telementoring in the surgical training of advanced laparoscopic operations. Surg Endosc. 1996;1:S75. 9. Bourke GJ, Daly LE, McGilvray J. Interpretation and Uses of Medical Statistics. Oxford, England: Blackwell Scientific Publications; 1985. 1. Goldstone LA. Understanding Medical Statistics. London, England: Heinemann Medical Books Ltd; 1983. 11. Squire D, Giachino AA, Profitt AW, Heaney C. Objective comparison of manual dexterity in physicians and surgeons. CJS. 1989;32:467-47. 12. Schueneman AL, Pickleman J, Hesselein R, et al. Neurological predictors of operative skill among general surgery residents. Surgery. 1984;96:288-295. 13. Lazar HL, DeLand EC, Tompkins RK. Clinical performance versus in-training examinations as measures of surgical competence. Surgery. 1989;87:357-362. 14. Barnes RW. Surgical handicraft: teaching and learning surgical skills. Am J Surg. 1987;153:422-427. 15. Martella AT, Santos GH. Priorities in general surgical training. Am J Surg. 1995; 169:271-272. 16. Forde KA. Minimal access surgery which path to competence? Surg Endosc. 1989;8:147-148. 17. Cuschieri A. A rose by any other name... : minimal access or minimally invasive surgery? Surg Endosc. 1992;6:214. 18. Cuschieri A. Shape of things to come: expectations and realism. Surg Endosc. 1994;8:83-85. 19. Scott-Conner CEH, Hall JT, Anglin BL, et al. The integration of laparoscopy into a surgical residency and implications for the training environment. Surg Endosc. 1994;8:154-157. 2. See WA, Cooper CS, Fisher RJ. Urological laparoscopic practice patterns 1 year after formal training. J Urol. 1994;151:1595-1598. 21. Coppola C, Merrell R. Effect of a training program on performance of laparoscopic cholecystectomy. Best Pract Benchmarking Healthcare. 1997;2:24-27. 22. Hayward CZ, Sachdeva A, Clarke JR. Is there gender bias in the evaluation of surgical residents? Surgery. 1987;12:297-299. 661