MENTORLEARN CLINICAL VALIDATION
The following abstract was presented at Annual Society of American Gastrointestinal and Endoscopic Surgeons (SAGES), April 2-5, 2014, Salt Lake City, Utah
Assessment of Medical Students Laparoscopic Skills Using Virtual Reality and Fundamentals of Laparoscopic Surgery Skills
Erica Sutton MD, Craig Ziegler MS, Kevin Martin BS, Stuart Crawford BS, Matthew Golden MD, J. David Richardson MD Hiram C. Polk Jr.
Department of Surgery, University of Louisville School of Medicine, Louisville, KY
Introduction: This study evaluates if undergraduate medical trainees’ laparoscopic skills acquisition could be monitored and assessed using a virtual reality (VR) simulator and how the resultant metrics correlate with actual performance of Fundamentals of Laparoscopic Surgery™ (FLS) tasks. A central tenet in creating competency-based curricula in undergraduate medical education is the development of meaningful assessments of medical student performance. Therefore, we studied the use of VR to characterize and assess the laparoscopic skills attained in a competency-based curriculum designed for graduating medical students applying for general surgery residency. Our overall goal is to integrate the milestone competencies for surgery across the educational continuum and document trainee progress toward proficiency.
Methods: Ten fourth year medical students applying for surgical residency completed a monitored virtual reality training curriculum comprised of camera navigation (CN), hand eye coordination (HEC) and FLS tasks: circle cutting (CC), ligating loop (LL), peg transfer (PT), and intracorporeal knot tying (IKT). Students completed the curriculum at their own pace over eight weeks. Performance goals were those of the simulator for CN and HEC tasks and the standard goals given in the FLS instruction manual for FLS tasks. Progress through the VR curriculum was monitored weekly by a single faculty member (FLS certified) using online software (Mentorlearn™).
After eight weeks, students were observed and recorded performing FLS tasks. The best VR performance for each FLS task and the observed performance of the FLS tasks were scored by assigning penalties as described in the FLS instruction manual. The ability of the VR simulator to detect penalties in each of the FLS tasks as well as correlations of time taken to complete tasks are reported. Additional metrics from the VR simulator (speed, number of movements, path length, strain, needle drops passes, and loading time) were examined for correlation to the commission of penalties.
Results: All ten students trained in 100% of the curriculum, though no student achieved proficiency in all of the VR modules assigned by the end of eight weeks. All students were proficient in CN and HEC tasks. Proficiency was achieved in CC, LL PT and IKT by 8, 6, 8, and 1 student respectively.
VR simulation showed high specificity for predicting zero penalties on the observed CC, LL, and PT tasks (78%, 80%, and 60%). VR ability to predict IKT penalties was most sensitive at two VR penalties (75%). VR consistently underestimates time for CC, LL, PT, and IKT tasks. Speed, number of movements, path length, strain, needle drops, passes, time needle is out of view and loading time did not correlate with penalties for any task
Discussion: At the University of Louisville, we are implementing curricula to improve the readiness of our undergraduate medical students to enter surgical training. This competency-based curriculum in laparoscopic skills is part of the Acting Internship in Surgery elective offered by the School of Medicine. In 2011, Edelman et al. found that medical students who trained on FLS tasks had a durable improvement of laparoscopic skills during their intern year when compared to interns who had not trained prior to entering internship and recommended the integration of FLS skills into undergraduate medical education.1,2 Our study uses performance of FLS tasks to assign meaning to assessments of medical students made on VR simulators for such curricula.
Conclusion: VR can be used to monitor and assess medical student acquisition of laparoscopic skills. The absence of penalties in the simulator reasonably predicts the absence of penalties in manual demonstration of CC, LL and PT skills, but not IKT. The documented skills acquired by trainees can be transferred to a graduate medical education program for further monitoring of progress toward proficiency.
1. Edelman DA, Mattos MA, Bouwman DL. Impact of fundamentals of laparoscopic surgery training during medical school on performance by first year surgical residents. J Surg Res. 2011 Sep;170(1):6-9.
2. Edelman DA, Mattos MA, Bouwman DL. Value of fundamentals of laparoscopic surgery training in a fourth-year medical school advanced surgical skills elective. J Surg Res. 2012 Oct;177(2):207-10.