RobotiX Mentor Clinical Validations
Training in Robotic Surgery: Initial Experience Using the Brazilian College of Surgeons Model
Fernando de Barros, Veronica Bernardino Felicio, Ana Caroline Lima Tabet, Ana Carolina Capuano Cerbone.
Hospital São Lucas, Department of General Surgery, Rio de Janeiro, Brasil.
Rev Col Bras Cir. 2021 Jun 14. https://doi.org/10.1590/0100-6991e-20202969
OBJECTIVE: To present the initial experience of the first tier of surgeons trained in the new model of robotic surgery training proposed by the CBC.
Methods: We retrospectively collected data and information on training with the Da Vinci SI robotic system. The variables analyzed were, in the pre-clinical phase, time of completion of each step by surgeon and number of hours in the simulator, and in the clinical phase, operations carried out by the training group, number of surgeons who performed nine procedures in ninety days (“9 in 90”), time of docking, time of console, and results surgical.
RESULTS: We interviewed 39 surgeons before training started; 20 (51.3%) reached the clinical phase. The average age of surgeons was 47.9 years (38-62). The average time between the first interview and the delivery of the online certificate was 64 days (15-133). The surgeons have made an average of 51h and 36 minutes of robot simulation (40-83 hours). The total number of cases in which the training surgeons participated as first assistant was 418, with an average of 20.9 per surgeon. The time of pre-clinical training had an average of 116 days (48-205).
CONCLUSION: The new model proposed had good acceptance by all surgeons trained and proved safe in the initial sample.
Validation of a Novel Simulation-Based Test in Robot-Assisted Radical Prostatectomy
Olsen RG, Bjerrum F, Konge L, Jepsen JV, Azawi NH, Bube SH.
Copenhagen Academy for Medical Education and Simulation (CAMES), Copenhagen, Denmark.
J Endourol. 2021 Mar 10. https://doi.org/10.1089/end.2020.0986
PURPOSE: To investigate validity evidence for a simulator-based test in robot-assisted radical prostatectomy (RARP).
MATERIALS AND METHODS: The test consisted of three modules on the RobotiX Mentor VR-simulator: Bladder Neck Dissection, Neurovascular Bundle Dissection, and Ureterovesical Anastomosis. Validity evidence was investigated by using Messick’s framework by including doctors with different RARP experience: novices (who had assisted for RARP), intermediates (robotic surgeons, but not RARP surgeons), or experienced (RARP surgeons). The simulator metrics were analyzed, and Cronbach’s alpha and generalizability theory were used to explore reliability. Intergroup comparisons were done with mixed-model, repeated measurement analysis of variance and the correlation between the number of robotic procedures and the mean test score were examined. A pass/fail score was established by using the contrasting groups’ method.
RESULTS: Ten novices, 11 intermediates, and 6 experienced RARP surgeons were included. Six metrics could discriminate between groups and showed acceptable internal consistency reliability, Cronbach’s alpha = 0.49, p < 0.001. Test–retest reliability was 0.75, 0.85, and 0.90 for one, two, and three repetitions of tests, respectively. Six metrics were combined into a simulator score that could discriminate between all three groups, p = 0.002, p < 0.001, and p = 0.029 for novices vs intermediates, novices vs experienced, and intermediates vs experienced, respectively. Total number of robotic operations and the mean score of the three repetitions were significantly correlated, Pearson's r = 0.74, p < 0.001. CONCLUSION: This study provides validity evidence for a simulator-based test in RARP. We determined a pass/fail level that can be used to ensure competency before proceeding to supervised clinical training.
Procedural virtual reality simulation training for robotic surgery: a randomized controlled trial
Nicholas Raison 1, Patrick Harrison 2, Takashige Abe 3, Abdullatif Aydin 2, Kamran Ahmed 2, Prokar Dasgupta 2
Affiliations expand
PMID: 33398587 DOI: 10.1007/s00464-020-08197-w
BACKGROUND: Virtual reality (VR) training is widely used for surgical training, supported by comprehensive, high-quality validation. Technological advances have enabled the development of procedural-based VR training. This study assesses the effectiveness of procedural VR compared to basic skills VR in minimally invasive surgery.
METHODS: 26 novice participants were randomised to either procedural VR (n = 13) or basic VR simulation (n = 13). Both cohorts completed a structured training programme. Simulator metric data were used to plot learning curves. All participants then performed parts of a robotic radical prostatectomy (RARP) on a fresh frozen cadaver. Performances were compared against a cohort of 9 control participants without any training experience. Performances were video recorded and assessed blindly using GEARS post hoc.
RESULTS: Learning curve analysis demonstrated improvements in technical skill for both training modalities although procedural training was associated with greater training effects.
Any VR training resulted in significantly higher GEARS scores than no training (GEARS score 11.3 ± 0.58 vs. 8.8 ± 2.9, p = 0.002). Procedural VR training was found to be more effective than both basic VR training and no training (GEARS 11.9 ± 2.9 vs. 10.7 ± 2.8 vs. 8.8 ± 1.4, respectively, p = 0.03).
CONCLUSIONS: This trial has shown that a structured programme of procedural VR simulation is effective for robotic training with technical skills successfully transferred to a clinical task in cadavers. Further work to evaluate the role of procedural-based VR for more advanced surgical skills training is required.
Development and validation of non‐guided bladder‐neck and neurovascular‐bundle dissection modules of the RobotiX‐Mentor full‐procedure robotic‐assisted radical prostatectomy virtual reality simulation
Jan Ebbing Peter N. Wiklund Olof Akre Stefan Carlsson Mats J. Olsson Jonas Höijer Maurice Heimer Justin W. Collins
The International Journal of Medical Robotics and Computer Assisted Surgery
BACKGROUND: Full‐procedure virtual reality (VR) simulator training in robotic‐assisted radical prostatectomy (RARP) is a new tool in surgical education.
METHODS: Description of the development of a VR RARP simulation model, (RobotiX‐Mentor®) including non‐guided bladder neck (ngBND) and neurovascular bundle dissection (ngNVBD) modules, and assessment of face, content, and construct validation of the ngBND and ngNVBD modules by robotic surgeons with different experience levels.
RESULTS: Simulator and ngBND/ngNVBD modules were rated highly by all surgeons for realism and usability as training tool. In the ngBND‐task construct validation was not achieved in task‐specific performance metrics. In the ngNVBD task‐specific performance of the expert/intermediately experienced surgeons was significantly better than that of novices.
CONCLUSIONS: We proved face and content validity of simulator and both modules, and construct validity for generic metrics of the ngBND module and for generic and task‐specific metrics of the ngNVBD module.
Design and validation of a cross‐specialty simulation‐based training course in basic robotic surgical skills
Susanne I. Scott Torur Dalsgaard Jan Vibjerg Jepsen Christian von Buchwald Steven Arild Wuyts Andersen
The International Journal of Medical Robotics and Computer Assisted Surgery
BACKGROUND: The aim of this study was to design and validate a cross‐specialty basic robotic surgical skills training program on the RobotiX Mentor virtual reality simulator.
METHODS: A Delphi panel reached consensus on six modules to include in the training program. Validity evidence was collected according to Messick’s framework with three performances in each simulator module by 11 experienced robotic surgeons and 11 residents without robotic surgical experience.
RESULTS: For five of the six modules, a compound metrics‐based score could significantly discriminate between the performances of novices and experienced robotic surgeons. Pass/fail levels were established, resulting in very few novices passing in their first attempt.
CONCLUSIONS: This validated course can be used for structured simulation‐based basic robotic surgical skills training within a mastery learning framework where the individual trainee can practice each module until they achieve proficiency and can continue training on other modalities and more specific to their specialty.
Assessment of validity evidence for the RobotiX robot assisted surgery simulator on advanced suturing tasks
Erik Leijte1,2* , Ivo de Blaauw2, Camiel Rosman1 and Sanne M. B. I. Botden2
BACKGROUND: Robot assisted surgery has expanded considerably in the past years. Compared to conventional open or laparoscopic surgery, virtual reality (VR) training is an essential component in learning robot assisted surgery. However, for tasks to be implemented in a curriculum, the levels of validity should be studied for proficiency-based training. Therefore, this study was aimed to assess the validity evidence of advanced suturing tasks on a robot assisted VR simulator.
METHOD: Participants were voluntary recruited and divided in the robotic experienced, laparoscopic experienced or novice group, based on self-reported surgical experience. Subsequently, a questionnaire on a five-point Likert scale was completed to assess the content validity. Three component tasks of complex suturing were performed on the RobotiX simulator (Task1: tilted plane needle transfer, Task: 2 intracorporal suturing, Task 3: anastomosis needle transfer). Accordingly, the outcome of the parameters was used to assess construct validity between robotic experienced and novice participants. Composite scores (0–100) were calculated from the construct parameters and corresponding pass/fail scores with false positive (FP) and false negative (FN) percentages.
RESULTS: Fifteen robotic experienced, 26 laparoscopic experienced and 29 novices were recruited. Overall content validity outcomes were scored positively on the realism (mean 3.7), didactic value (mean 4.0) and usability (mean 4.2). Robotic experienced participants significantly outperformed novices and laparoscopic experienced participants on multiple parameters on all three tasks of complex suturing. Parameters showing construct validity mainly consisted of movement parameters, needle precision and task completion time. Calculated composite pass/fail scores between robotic experienced and novice participants resulted for Task 1 in 73/100 (FP 21%, FN 5%), Task 2 in 85/100 (FP 28%, FN 4%) and Task 3 in 64/100 (FP 49%, FN 22%).
CONCLUSION: This study assessed the validity evidence on multiple levels of the three studied tasks. The participants score the RobotiX good on the content validity level. The composite pass/fail scores of Tasks 1 and 2 allow for proficiency-based training and could be implemented in a robot assisted surgery training curriculum.
Training Novice Robot Surgeons: Proctoring Provides Same Results As Simulator‑Generated Guidance
- J. W. Beulens, Y. A. F. Hashish, W. M. Brinkman, P. Umari, S. Puliatti, E. L. Koldewijn, A. J. M. Hendrikx, J. P. van Basten, J. J. G. van Merriënboer, H. G. Van der Poel, C. H. Bangma, C. Wagner. Department of Urology, Catharina Hospital, Eindhoven, The Netherlands Journal of Robotic Surgery
https://doi.org/10.1007/s11701-020-01118-y
To understand the influence of proctored guidance versus simulator generated guidance (SGG) on the acquisition dexterity skills in novice surgeons learning RAS (robot assisted surgery). Prospective non-blinded 3-arm randomised controlled trial (RTC). Exclusion criteria: previous experience in RAS or robotic surgery simulation.
The participants were assigned to three different intervention groups and received a different form of guidance: (1) proctored guidance, (2) simulator generated guidance, (3) no guidance, during training on virtual reality (VR) simulator. All participants were asked to complete multiple questionnaires. The training was the same in all groups with the exception of the intervention part. Catharina Hospital Eindhoven, The Netherlands. A total of 70 Dutch medical students, PhD-students, and surgical residents were included in the study. The participants were randomly assigned to one of the three groups.
Overall, all the participants showed a significant improvement in their dexterity skills after the training. There was no significant difference in the improvement of surgical skills between the three different intervention groups. The proctored guidance group reported a higher participant satisfaction compared to the simulator-generated guidance group, which could indicate a higher motivation to continue the training. This study showed that novice surgeons. Significantly increase their dexterity skills in RAS after a short time of practicing on simulator.
The lack of difference in results between the intervention groups could indicate there is a limited impact of “human proctoring” on dexterity skills during surgical simulation training. Since there is no difference between the intervention groups the exposure alone of novice surgeons to the robotic surgery simulator could possibly be sufficient to achieve a significant improvement of dexterity skills during the initial steps of RAS learning.
Training benchmarks based on validated composite scores for the RobotiX robot-assisted surgery simulator on basic tasks
Erik Leijte, Linda Claassen, Elke Arts, Ivo de Blaauw, Camiel Rosman, Sanne Botden, Department of Surgery, Radboud University Medical Centre, Nijmegen, The Netherlands
BACKGROUND: The RobotiX robot-assisted virtual reality simulator aims to aid in the training of novice surgeons outside of the operating room. This study aimed to determine the validity evidence on multiple levels of the RobotiX simulator for basic skills.
METHOD: Participants were divided in either the novice, laparoscopic or robotic experienced group based on their minimally invasive surgical experience. Two basic tasks were performed: wristed manipulation (Task 1) and vessel energy dissection (Task 2). The performance scores and a questionnaire regarding the realism, didactic value, and usability were gathered (content). Composite scores (0–100), pass/fail values, and alternative benchmark scores were calculated.
RESULTS: Twenty-seven novices, 21 laparoscopic, and 13 robotic experienced participants were recruited. Content validity evidence was scored positively overall. Statistically significant differences between novices and robotic experienced participants (construct) was found for movements left (Task 1 p = 0.009), movements right (Task 1 p = 0.009, Task 2 p = 0.021), path length left (Task 1 p = 0.020), and time (Task 1 p = 0.040, Task 2 p < 0.001). Composite scores were statistically significantly different between robotic experienced and novice participants for Task 1 (85.5 versus 77.1, p = 0.044) and Task 2 (80.6 versus 64.9, p = 0.001). The pass/fail score with false-positive/false-negative percentage resulted in a value of 75/100, 46/9.1% (Task 1) and 71/100, 39/7.0% (Task 2). CONCLUSION: Calculated benchmark scores resulted in a minority of novices passing multiple parameters. Validity evidence on multiple levels was assessed for two basic robot-assisted surgical simulation tasks. The calculated benchmark scores can be used for future surgical simulation training. Read more
Concurrent, face, content, and construct validity of the RobotiX Mentor simulator for robotic basic skills
Muaath Alshuaibi, Cyril Perrenot. Jacques Hubert, Manuela Perez.
School of Surgery, Faculty of Medicine, Lorraine University, Vandoeuvre-lès-Nancy, France
https://onlinelibrary.wiley.com/doi/full/10.1002/rcs.2100
OBJECTIVE: To assess several criteria, such as concurrent, face, content, and construct validity of the RobotiX Mentor (RXM) simulator for basic robotic skills and to compare virtual and actual dry lab dome.
METHODS: A prospective study was conducted from December 2017 to May 2018 using RXM and a da Vinci Si robot. 37 subjects, divided into three groups according to their initial surgical training (expert, intermediate, and novice), were evaluated in terms of six representative exercises of basic robotic specific skills as recommended by the fundamentals of robotic surgery.
RESULTS: There was a correlation between the automatic data from the RXM and the subjective evaluation with the robot. The face and content validity, which were evaluated by the experts, were generally considered high (71.5% and 62.5%, respectively).Three levels (analysis of variance [ANOVA]; P = .01) and two levels (P = .001) of experience were clearly identified by the simulator.
CONCLUSION: Our study proves the concurrent validity and confirms the face, content, and construct validity of the RXM.
Demonstrating the Effectiveness of the Fundamentals of Robotic Surgery (FRS) on the RobotiX Mentor Virtual Reality Simulation Platform
JR Martin, D. Stefanidis, RP. Dorin, AC. Goh, RM. Satava, J. Levy.
Center for Education, Simulation and Innovation, Hartford Hospital, Hatford, CT.
Methodist Institute for Technology, Innovation, and Education (MITIE), Houston Methodist Hospital, Houston, TX.
STUDY OBJECTIVE: To determine if robotic surgery novices demonstrate improved technical skill after completing Fundamentals of Robotic Surgery (FRS) proficiency-based psychomotor skills training using the RobotiX Mentor virtual reality (VR) platform.
DESIGN: An observational, pre-post design, multi-institutional rater-blinded trial was conducted. Post-hoc comparisons were performed against previously published comparator groups.
SETTING: Robotic surgery training facilities at the Methodist Institute for Technology, Innovation, & Education (Houston, TX) and the Center for Education, Simulation & Innovation at Hartford Hospital (Hartford, CT) were used for this study.
PATIENTS OR PARTICIPANTS: Residents (n=20) who were robotic surgery novices were enrolled to participate.
INTERVENTIONS: Participants completed FRS online didactic modules and were required to pass the FRS cognitive exam. Participants’ baseline robotic surgery skills were assessed on an avian tissue model with previously published validity evidence (pre-test). Participants then trained using the FRS proficiency-based progression curriculum on the RobotiX Mentor VR platform. Participants were required to perform each task to expert-derived benchmarks on consecutive attempts before proceeding to the next task. After training completion, participants were tested again on the avian tissue model (post-test). Pre- and post-tests were video recorded and assessed by blinded raters using the Global Evaluative Assessment of Robotic Skills (GEARS) and a 32-criteria psychomotor checklist.
MEASUREMENTS AND MAIN RESULTS: On paired-samples T tests, participants demonstrated improved performance from pre- to post-test across all GEARS domains (depth perception, bimanual dexterity, efficiency, use of force, instrument control; p=0.01 to p<0.001) and for time (p<0.001) and errors (p=0.003) as measured by psychometric checklist. By ANOVA, improvement in novices’ robotic skill after training on the RobotiX Mentor was not inferior to improvement reported after training on two currently-available VR platforms. CONCLUSION: Completion of the FRS curriculum on the RobotiX Mentor VR training platform resulted in improved robotic surgery skills among novices, proving effectiveness of training. The RobotiX Mentor is recommended as a training platform for the FRS curriculum
Robot Assisted Versus Laparoscopic Suturing Learning Curve in a Simulated Setting
Erik Leijte, Ivo de Blaauw, Frans Van Workum, Camiel Rosman, Sanne Botden, Department of Surgery, Radboud University Medical Centre, Nijmegen, The Netherlands
BACKGROUND: Compared to conventional laparoscopy, robot assisted surgery is expected to have most potential in difficult areas and demanding technical skills like minimally invasive suturing. This study was performed to identify the differences in the learning curves of laparoscopic versus robot assisted suturing.
METHOD: Novice participants performed three suturing tasks on the EoSim laparoscopic augmented reality simulator or the RobotiX robot assisted virtual reality simulator. Each participant performed an intracorporeal suturing task, a tilted plane needle transfer task and an anastomosis needle transfer task. To complete the learning curve, all tasks were repeated up to twenty repetitions or until a time plateau was reached. Clinically relevant and comparable parameters regarding time, movements and safety were recorded. Intracorporeal suturing time and cumulative sum analysis was used to compare the learning curves and phases.
RESULTS: Seventeen participants completed the learning curve laparoscopically and 30 robot assisted. Median frst knot suturing time was 611 s (s) for laparoscopic versus 251 s for robot assisted (p<0.001), and this was 324 s versus 165 (sixth knot, p<0.001) and 257 s and 149 s (eleventh knot, p<0.001) respectively on base of the found learning phases. The percentage of ‘adequate surgical knots’ was higher in the laparoscopic than in the robot assisted group. First knot: 71% versus 60%, sixth knot: 100% versus 83%, and eleventh knot: 100% versus 73%. When assessing the ‘instrument out of view’ parameter, the robot assisted group scored a median of 0% after repetition four. In the laparoscopic group, the instrument out of view increased from 3.1 to 3.9% (left) and from 3.0 to 4.1% (right) between the frst and eleventh knot (p>0.05).
CONCLUSION: The learning curve of minimally invasive suturing shows a shorter task time curve using robotic assistance compared to the laparoscopic curve. However, laparoscopic outcomes show good end results with rapid outcome improvement.
A Prospective Study of the Effect of Video Games on Robotic Surgery Skills Using the High-Fidelity Virtual Reality RobotiX Simulator
Andreas Pierre Hvolbek, Philip Mørkeberg Nilsson, Francesco Sanguedolce, Lars Lund
Aalborg University Hospital and Copenhagen Academy for Medical Education and Simulation, Rigshospitalet, Copenhagen, Denmark
DOI: https://doi.org/10.2147/AMEP.S199323
BACKGROUND: Robot-assisted surgery is a growing field. Prior video game experience might give advantage to novice robotic surgeons.
AIM: Assessing if prior video gaming experience gives advantage in performing high-fidelity virtual reality (VR)-simulated robotic surgery.
METHODS: In this observational study, 30 medical students and 2 interns (17 females; 15 males) with median age 25 years (range, 24–26 years) were recruited and subsequently divided into groups according to prior gaming experience; gamers (≥6 video game hours/ week) vs nongamers (<6 video game hours/week). Participants performed VR-simulated urethrovesical anastomosis on RobotiX Mentor, which measured performance parameters. Participants answered a questionnaire for demographics and gaming experience. Groups were compared using Mann–Whitney U and multiple regression. RESULTS: Gamers significantly outperformed nongamers in 3 of 24 performance metrics (p<0.05(, and there was a trend toward better results for 7 of the 21 remaining metrics. Males outperformed females in 5 of 24 metrics (p<0.05) but were overrepresented among gamers. CONCLUSION: Prior video game experience >6 hrs/week might give advantage in simulated robotic surgery. We recommend future studies testing this hypothesis to develop simulator programs for certification of robotic surgeons.
Evaluation of procedural virtual reality simulation training: a randomised controlled trial
Nicholas Raison, Patrick Harrison, London, United Kingdom; Takashige Abe, Sapporo, Japan; Abdullatif Aydin, Senthil Nathan, Shamim Khan, Kamran Ahmed, Prokar Dasgupta, London, United Kingdom
Published in the JOURNAL OF UROLOGY
https://www.auajournals.org/doi/pdf/10.1097/01.JU.0000555974.72575.72
INTRODUCTION AND OBJECTIVES: Improvements in virtual reality (VR) technology have enabled the development of procedural simulation training which closely replicate surgical procedures. VR simulation training has been shown to be highly effective for robotic surgical training however to-date curricula are limited to generic basic skills training. This RCT aims to compare the transfer of learning following procedural VR or standard basic skills VR training.
METHODS: Initially 25 novice surgeons underwent basic robotic skills training, completing three FRS tasks. Participants were then block randomised to standard basic VR training or procedural VR training. All training was performed on the RobotiX Mentor (Surgical Science, Airport City, Israel) VR robotic simulator. Standard basic skills training comprised further training following the FRS curriculum. The procedural simulation group underwent training on the guided bladder neck dissection and guided urethrovesical anastomosis tasks, parts of radical prostatectomy training module. Both groups completed a total of at least 5 hours of training. Following training both groups underwent transfer of skills assessment on fresh frozen cadavers using a Da Vinci Xi surgical robot in a simulated operating room environment. Their performances were compared to a control group of novice training without training. All performances were video recorded and were assessed blindly post hoc by a trained expert using GEARS.
RESULTS: Baseline FRS scores were equal between the two groups (p[0.5). Subjects in both arms completed an average of 5.6 0.3 hours of training. VR training (basic or procedural) resulted in a significantly higher GEARS score than no training, (mean GEARS score 11.3 0.6 vs 8.8 2.9 p[0.002). Procedural training resulted in significantly higher GEARS score than either basic training or control (p[0.03)(Figure 1).
CONCLUSIONS: This study provides the first evidence supporting the use of procedural-based VR simulation for training robotic skills even in novice participants. It also provides further validity evidence to support the use of VR simulation and the effective transfer of learned skills
Head-to-Head Comparison of Three Virtual-Reality Robotic Surgery Simulators
Alexandria M. Hertz, MD, Evalyn I. George, BS, Christine M. Vaccaro, DO, Timothy C. Brand, MD
Published in JSLS : Journal of the Society of Laparoendoscopic Surgeons 22:e2017.00081
BACKGROUND AND OBJECTIVES: There are several different commercially available virtual-reality robotic simulators, but very little comparative data. We compared the face and content validity of 3 robotic surgery simulators and their pricing and availability.
METHODS: Fifteen participants completed one task on each of the following: dV-Trainer (dVT; Mimic Technologies, Inc., Seattle, Washington, USA), da Vinci Skills Simulator (dVSS; Intuitive Surgical Inc., Sunnyvale, California, USA), and RobotiX Mentor (RM; Surgical Science). Participants completed previously validated face and content validity questionnaires and a demographics questionnaire. Statistical analysis was then performed on the scores.
RESULTS: Participants had a mean age of 29.6 (range, 25– 41) years. Most were surgical trainees, having performed a mean of 8.6 robotic primary surgeries. For face validity, ANOVA showed a significant difference favoring the dVSS over the dVT (P .001), and no significant difference between the RM, dVSS, and dVT. Content validity revealed similar results, with a significant difference between the dVSS and dVT (P .021), a trend toward a difference between the RM and dVT (P .092), and no difference between the dVSS and RM (P .99).
CONCLUSION: All simulators demonstrated evidence of face and content validity, with significantly higher scores for the dVSS; it is also the least costly ($80,000 for the simulator), although it is frequently unavailable because of intra-operative use. The dVT and RM have similar face and content validity, are slightly more expensive, and are readily available.
Validity assessment of a simulation module for robot-assisted thoracic lobectomy
BACKGROUND:Training for robot-assisted thoracic lobectomy remains an issue, prompting the development of virtual reality simulators. Our aim was to assess the construct and face validity of a new thoracic lobectomy module on the RobotiX Mentor, a robotic surgery simulator. We also aimed to determine the acceptability and feasibility of implementation into training.
METHODS:This prospective, observational, and comparative study recruited novice (n = 16), intermediate (n = 9), and expert (n = 5) participants from King’s College London, the 25th European Conference on General Thoracic Surgery, and the Society of Robotic Surgery conference 2018. Each participant completed two familiarization tasks followed by the Guided Robotic Lobectomy module and an evaluation questionnaire. Outcome measures were compared using Mann-Whitney U tests.
RESULTS:Construct validity was demonstrated in 12/21 performance evaluation metrics. Significant differences between groups were found in all metrics including: time taken to complete module, vascular injury, respect for tissue, number of stapler firings, time instruments out of view, number of instrument collisions, and number of movements. Participants deemed aspects of the simulator (mean 3/5) and module (3/5) as realistic and rated the simulator as both acceptable (3.8/5) and feasible (3.8/5) for robotic surgical training.
CONCLUSION: Face validity, acceptability, and feasibility were established for the thoracic lobectomy module of the RobotiX Mentor simulator. Moderate evidence of construct validity was also demonstrated. With further work, this simulation module could help to reduce the initial part of the learning curve for trainees and decrease the risk of errors during live training.
Establishing objective benchmarks in robotic virtual reality simulation at the level of a competent surgeon using the RobotiX Mentor simulator
William Watkinson, Nicholas Raison, Takashige Abe, Patrick Harrison, Shamim Khan, Henk Van der Poel, Prokar Dasgupta, Kamran Ahmed
Published: Postgraduate Medical Journal, Volume 94, Issue 1111
Published Online First 6 March 2018
http://dx.doi.org/10.1136/postgradmedj-2017-135351
BACKGROUND: To establish objective benchmarks at the level of a competent robotic surgeon across different exercises and metrics for the RobotiX Mentor virtual reality (VR) simulator suitable for use within a robotic surgical training curriculum.
METHODS: This retrospective observational study analysed results from multiple data sources, all of which used the RobotiX Mentor VR simulator. 123 participants with varying experience from novice to expert completed the exercises. Competency was established as the 25th centile of the mean advanced intermediate score. Three basic skill exercises and two advanced skill exercises were used.
SETTING: King’s College London.
PARTICIPANTS: 84 Novice, 26 beginner intermediates, 9 advanced intermediates and 4 experts were used in this retrospective observational study.
RESULTS: Objective benchmarks derived from the 25th centile of the mean scores of the advanced intermediates provided suitably challenging yet also achievable targets for training surgeons. The disparity in scores was greatest for the advanced exercises. Novice surgeons are able to achieve the benchmarks across all exercises in the majority of metrics.
CONCLUSION: We have successfully created this proof-of-concept study, which requires validation in a larger cohort. Objective benchmarks obtained from the 25th centile of the mean scores of advanced intermediates provide clinically relevant benchmarks at the standard of a competent robotic surgeon that are challenging yet also attainable. That can be used within a VR training curriculum allowing participants to track and monitor their progress in a structured and progressional manner through five exercises. Providing clearly defined targets, ensuring that a universal training standard has been achieved across training surgeons.
Validity evidence for procedural competency in virtual reality robotic simulation, establishing a credible pass/fail standard for the vaginal cuff closure procedure
Lisette Hvid Hovgaard, Steven Arild Wuyts Andersen, Lars Konge, Torur Dalsgaard, Christian Rifbjerg Larsen
Published in Surgical Endoscopy, March 2018
DOI: 10.1007/s00464-018-6165-5
OBJECTIVE: The use of robotic surgery for minimally invasive procedures has increased considerably over the last decade. Robotic surgery has potential advantages compared to laparoscopic surgery but also requires new skills. Using virtual reality (VR) simulation to facilitate the acquisition of these new skills could potentially benefit training of robotic surgical skills and also be a crucial step in developing a robotic surgical training curriculum. The study’s objective was to establish validity evidence for a simulation-based test for procedural competency for the vaginal cuff closure procedure that can be used in a future simulation-based, mastery learning training curriculum.
METHODS: Eleven novice gynaecological surgeons without prior robotic experience and 11 experienced gynaecological robotic surgeons (> 30 robotic procedures) were recruited. After familiarization with the VR simulator, participants completed the module ‘Guided Vaginal Cuff Closure’ six times. Validity evidence was investigated for 18 preselected simulator metrics. The internal consistency was assessed using Cronbach’s alpha and a composite score was calculated based on metrics with significant discriminative ability between the two groups. Finally, a pass/fail standard was established using the contrasting groups’ method.
RESULTS: The experienced surgeons significantly outperformed the novice surgeons on 6 of the 18 metrics. The internal consistency was 0.58 (Cronbach’s alpha). The experienced surgeons’ mean composite score for all six repetitions were significantly better than the novice surgeons’ (76.1 vs. 63.0, respectively, p < 0.001). A pass/fail standard of 75/100 was established. Four novice surgeons passed this standard (false positives) and three experienced surgeons failed (false negatives.)
CONCLUSION: Our study has gathered validity evidence for a simulation-based test for procedural robotic surgical competency in the vaginal cuff closure procedure and established a credible pass/fail standard for future proficiency-based training.
The Validation of a Novel Robot-Assisted Radical Prostatectomy Virtual Reality Module
Patrick Harrison, BSc, Nicholas Raison, MRCS, Takashige Abe, PhD, MD, William Watkinson, BSc, Faizan Dar, MBBS, Ben Challacombe, MS, FRCS(Urol), Henk Van Der Poel, MD, PhD, Muhammad Shamim Khan, FRCS(Urol), Prokar Dasgupa, MD, FEBU, FRCS(Urol), Kamran Ahmed, FRCS(Urol), Ph
Published online:September 30, 2017
OBJECTIVE: To perform the first validation of a full procedural virtual reality robotic training module and analysis of novice surgeon’s learning curves.
DESIGN: Participants completed the bladder neck dissection task and urethrovesical anastomosis task (UVA) as part of the prostatectomy module. Surgeons completed feedback questionnaires assessing the realism, content, acceptability and feasibility of the module. Novice surgeons completed a 5.5-hour training programme using both tasks.
SETTING: King’s College London, London.
PARTICIPANTS: 13 novice, 24 intermediate and 8 expert surgeons completed the validation study.
RESULTS: Realism was scored highly for BDN (mean 3.4/5) and UVA (3.74/5), as was importance of BDN (4.32/5) and UVA (4.6/5) for training. It was rated as a feasible (3.95/5) and acceptable (4/5) tool for training. Experts performed significantly better than novice group in 6 metrics in the UVA including time (p = 0.0005), distance by camera (p = 0.0010) and instrument collisions (p = 0.0033), as well as task-specific metrics such as number of unnecessary needle piercing points (p = 0.0463). In novice surgeons, a significant improvement in performance after training was seen in many metrics for both tasks. For bladder neck dissection task, this included time (p < 0.0001), instrument collisions (p = 0.0013) and total time instruments are out of view (p = 0.0251). For UVA, this included time (p = 0.0135), instrument collisions (p = 0.0066) and task-specific metrics such as injury to the urethra (p = 0.0032) and bladder (p = 0.0189).
CONCLUSIONS: Surgeons found this full procedural VR training module to be a realistic, feasible and acceptable component for a robotic surgical training programme. Construct validity was proven between expert and novice surgeons. Novice surgeons have shown a significant learning curve over 5.5 hours of training, suggesting this module could be used in a surgical curriculum for acquisition of technical skills. Further implementation of this module into the curriculum and continued analysis would be beneficial to gauge how it can be fully utilised.
A comparative analysis and guide to virtual reality robotic surgical simulators
Julian D, Tanaka A, Mattingly P, Truong M, Perez M, Smith R
WILEY, The International Journal of Medical Robotics and Computer Assisted Surgery
Accepted: 9 October 2017; DOI: 10.1002/rcs.1874
BACKGROUND: Since the US Food and Drug Administration approved robotically assisted surgical devices for human surgery in 2000, the number of surgeries utilizing this innovative technology has risen. In 2015, approximately 650 000 robot-assisted procedures were performed worldwide. Surgeons must be properly trained to safely transition to using such innovative technology. Multiple virtual reality robotic simulators are now commercially available for educational and training purposes. There is a need for comparative evaluations of these simulators to aid users in selecting an appropriate device for their purposes.
METHODS: We conducted a comparison of the design and capabilities of all dedicated simulators of the da Vinci robot – the da Vinci Skills Simulator (dVSS), dV-Trainer (dVT), Robotic Skills Simulators (RoSS) and the RobotiX Mentor. This paper provides the base specifications of the hardware and software, with an emphasis on the training capabilities of each system.
RESULTS: Each simulator contains a large number of training exercises for skills development: dVSS n = 40, dVT n = 65, RoSS n = 52, RobotiX Mentor n = 31. All four offer 3D visual images but use different display technologies. The dVSS leverages the real robotic surgical console to provide visualization, hand controls and foot pedals. The dVT, RoSS and RobotiX Mentor created simulated versions of all of these control systems. Each includes systems management services that allow instructors to collect, export and analyze the scores of students using the simulators.
CONCLUSIONS: This study provides comparative information on the four simulators’ functional capabilities. Each device offers unique advantages and capabilities for training robotic surgeons. Each has been the subject of validation experiments, which have been published in the literature. But those do not provide specific details on the capabilities of the simulators, which are necessary for an understanding sufficient to select the one best suited for an organization’s needs. This article provides comparative information to assist with that type of selection.
The Robotix Simulator: Face And Content Validation Using The Fundamentals Of Robotic Surgery (FRS) Curriculum
Ismail Omar*, James Dilley, Philip Pucher, Philip Pratt, Torath Ameen,
Justin Vale, Ara Darzi, Erik Mayer, London, United Kingdom
The Journal of Urology
Volume 197, Issue 4, Pages e700-e701 (April 2017)
DOI: 10.1016/j.juro.2017.02.1626
INTRODUCTION AND OBJECTIVES: Robotic surgical training is poorly delivered with limited skills-based training offered to trainees. To improve this, validated high quality robotic simulators need to be developed and made available. The RobotiX simulator is a new platform which this study sought to validate in the context of a recognised robotic surgery curriculum.
METHODs: Surgeons(n¼29) with ranging robotic experience and experience with other simulators(da Vinci Backpack and Mimic) were invited to complete all 6 FRS curriculum exercises. Participant performance was scored using the Global Evaluation Assessment of Robotic skills(GEARS). Participants completed a Likert scale based face and content validity questionnaire graded as negative(1-2/5), neutral(3), or positive(4-5).
RESULTS: Overall, analysis included 27 participants. There was good concurrent GEARS score reliability (Cronbach’s Alpha 0.801) between participants performing exercises on both the RobotiX and an alternative robotic simulator.
CONCLUSIONS: The RobotiX simulator demonstrated excellent Face and Construct validity evidence both in terms of general usage and in the specific context of the FRS curriculum.(Table 2 Q1,2)
The RobotiX performed at least as well as other simulators(da Vinci Backpack/Mimic) and was found to simulate a robotic platform such as the da Vinci Robot. (Table 2 Q4,5) Furthermore 81% of surgical trainees would recommend using the RobotiX(Table 2 Q3).
A methodological, task-based approach to Procedure-Specific Simulations training
Yaki Setty, Oren Salzman
International Journal of Computer Assisted Radiology and Surgery
December 2016, Volume 11, Issue 12, pp 2317–2324
PURPOSE: Procedure-Specific Simulations (PSS) are 3D realistic simulations that provide a platform to practice complete surgical procedures in a virtual-reality environment. While PSS have the potential to improve surgeons’ proficiency, there are no existing standards or guidelines for PSS development in a structured manner.
METHOD: We employ a unique platform inspired by game design to develop virtual reality simulations in three dimensions of urethrovesical anastomosis during radical prostatectomy. 3D visualization is supported by a stereo vision, providing a fully realistic view of the simulation. The software can be executed for any robotic surgery platform. Specifically, we tested the simulation under windows environment on the RobotiX Mentor.
RESULT: Using urethrovesical anastomosis during radical prostatectomy simulation as a representative example, we present a task-based methodological approach to PSS training. The methodology provides tasks in increasing levels of difficulty from a novice level of basic anatomy identification, to an expert level that permits testing new surgical approaches.
CONCLUSION: The modular methodology presented here can be easily extended to support more complex tasks. We foresee this methodology as a tool used to integrate PSS as a complementary training process for surgical procedures.
Validation of the RobotiX Mentor Robotic Surgery Simulator
Whittaker, A. Aydin, N. Raison, F. Kum, B. Challacombe, M.S. Khan,P. Dasgupta, K. Ahmed
MRC Centre for Transplantation, King’s College London; Department of Urology, Guy’s
and St. Thomas’ NHS Foundation Trust, London, United Kingdom
Proceedings of the Hamlyn Symposium on Medical Robotics
20-23 June 2015, Imperial College London London, UK
ISBN: 978-0-9563776-6-1 p 69-70
INTRODUCTION: With robotic-assisted surgery becoming more common practice in urology, effective training remains a challenge. There is a considerable learning curve associated with robotic training, though this has been reported as gentler than for laparoscopic training1.
Simulation has gained wide acceptance as a method of reducing the initial phase of the learning curve. The RobotiX MentorTM is a new virtual reality simulator which mimics the interface of the da Vinci® Surgical System, with integrated training modules including Fundamentals of Robotic Surgery (FRS) and Robotic Suturing. Face and content validity of the suturing module have been confirmed in a previous study2, though this was performed on a different platform. This study aims to assess face, content and construct validity of the RobotiX MentorTM. It also aims to assess its acceptability as a training tool and feasibility of its use in training.
MATERIALS AND METHODS: This prospective, observational and comparative study recruited novice (n=20), intermediate (n=15), and expert (n=11) robotic surgeons as participants from institutions
across the United Kingdom and at the 30th European Association of Urology Annual Meeting. Each participant completed nine surgical tasks across two modules on the simulator, followed by a questionnaire to evaluate subjective realism (face validity), task importance (content validity), feasibility, and acceptability. Outcome measures of novice, intermediate, and expert groups were compared using
Mann-Whitney U-tests to assess construct validity. Outcome measures of novice, intermediate, and expert groups were compared using Mann-Whitney U-tests to assess construct validity.
RESULTS: Construct validity was demonstrated in a total of 17/25 performance evaluation metrics (p<0.001). Experts performed better than intermediates in regard to time taken to complete the first (p=0.002) and second (p=0.043) module, number of instrument collisions (p=0.040), path length (p=0.049), number of cuts >2mm deep (p=0.033), average distance from suture target (p=0.015), and number of suture breakages (p=0.038). Participants determined both the simulator console and psychomotor tasks as highly realistic (mean: 3.7/5) and very important for surgical training (4.5/5), with system pedals (4.2/5) and knot tying task (4.6/5) scoring highest respectively. The simulator was also rated as an acceptable (4.3/5) tool for training and its use highly feasible (4.3/5).
CONCLUSIONS: Construct, face, and content validity were established for the RobotiX Mentor and feasibility and acceptability of incorporation into surgical training was ascertained. The RobotiX Mentor shows potential as a valuable tool for training and assessment of trainees in robotic skills and may reduce the initial learning curve if utilised as an adjunct to operating-room training. Investigation of
concurrent and predictive validity is necessary to complete validation and evaluation of learning curves
would provide insight into its value for training.
REFERENCES: [1] Goldstraw MA, Challacombe BJ, Patil K, Amoroso P, Dasgupta P, Kirby RS. Overcoming the challenges of robot-assisted radical prostatectomy. Prostate cancer and prostatic diseases. Mar 2012;15(1):1-7.
[2] Abboudi H, Khan MS, Aboumarzouk O, et al. Current status of validation for robotic surgery simulators-a systematic review. BJU international. Feb 2013;111(2):194-205. 69
From Design to Conception: An Assessment Device for Robotic Surgeons
Alyssa Tanaka, M.S.; Manuela Perez, M.D. Mireille Truong M.D., Khara Simpson M.D. Gareth Hearn, Roger Smith, Ph.D.
Interservice/Industry Training, Simulation, and Education Conference (I/ITSEC) 2014
ABSTRACT: The daVinci Surgical System offers surgeons improved capabilities for performing complex minimally invasive procedures; however, there is no standardized assessment of robotic surgeons and a need exists to ensure that a minimal standard of care is provided to all patients. The Department of Defense and governing surgical societies convened consensus conferences to develop a national initiative, resulting in a curriculum called the Fundamentals of Robotic Surgery (FRS). FRS is comprised of an online curriculum and a psychomotor skills dome.This paper describes the production process used to create a psychomotor skills assessment device – the FRS Dome. The device was designed to measure the essential skills that are required of any robotic surgeon and to provide a basis upon which to grant or deny privileging with the robot. It was constructed to test seven tasks of manual dexterity: Docking, Ring Tower Transfer, Knot Tying, Suturing, 4th Arm Cutting, Puzzle Piece Dissection, and Energy Dissection.
The initial design of the device was created by a committee of experienced minimally invasive surgeons, with a background in testing protocols and materials. The design was rendered in computer animation, which kickstarted a prototyping effort with physical materials. These included platinum cure silicone approximating human tissue and a 3D polyjet printer for the structural framework. Usability testing was conducted and iterative modifications were made to improve ergonomics, standardization, and cost requirements. Final CAD diagrams and specifications were created and distributed to medical and simulation companies for both physical and digital manufacturing. This development process demonstrates the evolution of a simulation and a physical testing device based on international expert consensus. The specifications are open source, allowing competitive production and future iterations. The goal of this paper is to discuss how this device evolved from an idea to a manufactured product and a digital simulation.
Current status of robotic simulators in acquisition of robotic surgical skills.
Kumar A, Smith R, Patel VR.
Department of Urology, Global Robotics Institute, Florida Hospital-Celebration Health Chief Technology Officer, Florida Hospital-Nicholson Center, University of Central Florida College of Medicine, Orlando, Florida, USA.
Curr Opin Urol. 2015 Mar;25(2):168-74.
PURPOSE OF REVIEW: This article provides an overview of the current status of simulator systems in robotic surgery training curriculum, focusing on available simulators for training, their comparison, new technologies introduced in simulation focusing on concepts of training along with existing challenges and future perspectives of simulator training in robotic surgery.
RECENT FINDINGS: The different virtual reality simulators available in the market like dVSS, dVT, RoSS, ProMIS and SEP have shown face, content and construct validity in robotic skills training for novices outside the operating room. Recently, augmented reality simulators like HoST, Maestro AR and RobotiX Mentor have been introduced in robotic training providing a more realistic operating environment, emphasizing more on procedure-specific robotic training . Further, the Xperience Team Trainer, which provides training to console surgeon and bed-side assistant simultaneously, has been recently introduced to emphasize the importance of teamwork and proper coordination.
SUMMARY: Simulator training holds an important place in current robotic training curriculum of future robotic surgeons. There is a need for more procedure-specific augmented reality simulator training, utilizing advancements in computing and graphical capabilities for new innovations in simulator technology. Further studies are required to establish its cost-benefit ratio along with concurrent and predictive validity.
Further studies are required to establish its cost-benefit ratio along with concurrent and predictive validity.
Surgical suturing training with virtual reality simulation versus dry lab practice: an evaluation of performance improvement, content, and face validity
Michael J. Amirian, Samuel M. Lindner, Edouard J. Trabulsi, Costas D. Lallas
Department of Urology, Thomas Jefferson University Hospital, 1015 Walnut Street, Suite 1100, Philadelphia, PA 19107, USA
J Robotic Surg (2014) 8:329–335
The purpose of this study is to evaluate the effectiveness of virtual reality (VR) simulation versus dry lab suturing practice at improving suturing performance in robotic surgery. Nineteen novice participants with no prior robotic suturing experience were randomized to two groups, VR simulation and dry lab, which consisted of inanimate training on a da Vinci Si surgical system. Each group underwent baseline suturing evaluation, then trained on the Simbionix™ Suturing Module (SSM) or undertook suturing practice using the da Vinci Surgical System in a dry lab. Final suturing performance was evaluated using the objective suture scoring method. Participants in the VR simulation group were surveyed to assess the face and content validity of the SSM. Both groups experienced significant improvement after training (VR simulation group p = 0.0078; dry lab group p = 0.0039). There was no significant difference in improvement between the two groups after undergoing training with either SSM or in the dry lab. Improvements in composite timing scores were 123 and 172 in the VR simulation and dry lab test groups, respectively (p = 0.36). Face validation varied with respect to the category assessed, but participants confirmed content validity of the SSM in all categories. In this sample of novice operators, there was no significant advantage in training with VR simulation using the SSM over dry lab training in improving suturing performance. Users of the SSM found it useful and relevant as a training tool for improving suturing performance.