ANGIO MENTOR CLINICAL VALIDATIONS
Simulation-Based Education of Endovascular Scrub Nurses Reduces Stress and Improves Team Performance
Skov RAC, Lawaetz J, Konge L, Westerlin L, Aasvang EK, Meyhoff CS, Vogt K, Ohrlander T, Resch TA, Eiberg JP. Simulation-Based Education of Endovascular Scrub Nurses Reduces Stress and Improves Team Performance. J Surg Res. 2022 Dec;280:209-217
- Department of Vascular Surgery, Rigshospitalet, Copenhagen, Denmark
- Department of Clinical Medicine, Faculty of Health and Medical Sciences, University of Copenhagen, Denmark, Copenhagen
- Academy for Medical Education and Simulation (CAMES), Denmark d Department of Anaesthesiology, Rigshospitalet, Copenhagen, Denmark
- Department of Anaesthesia and Intensive Care, Bispebjerg and Frederiksberg Hospital, Copenhagen, Denmark
Journal of Surgical Research, December 2022
INTRODUCTION: Endovascular procedures have become commonplace in vascular surgery. This development calls for new training strategies for future specialists. Most simulation-based education (SBE) programs have a monodisciplinary focus on physicians, although successful surgery is a multidisciplinary team effort. Mental stress impairs the learning process and surgical performance and heart rate variability (HRV) can be measured as a proxy for both mental and physical stress. This study aims to assess how SBE of endovascular scrub nurses affects team performance and HRV during endovascular aneurysm repair (EVAR).
MATERIALS AND METHODS: Prospective interventional study in which EVAR-inexperienced scrub nurses followed a focused SBE EVAR program. During real-life EVAR procedures, HRV was continuously recorded with a wireless electrocardiogram patch and multidisciplinary team performance was assessed with the Imperial College Error CAPture (ICECAP) tool, before and after the SBE program, allowing each scrub nurse to serve as their own control. Eight scrub nurses with experience in lower limb endovascular procedures, but not EVAR, were invited to participate.
RESULTS: Seven participants completed the study. In five of seven scrub nurses, HRV-derived stress levels during real-time EVAR procedures were lower after SBE compared to before SBE. Mean HRV increased from 24 msec to 35 msec (P < 0.001), indicating stress level reduction. Before SBE, the mean number of errors/hour was 7.3 (standard deviation ± 1.8) compared to 3.6 (standard deviation ± 2.7) after SBE. Most errors were categorized as technical (58 %) and communicative (23 %).
CONCLUSIONS: SBE of scrub nurses may improve team performance and may lower mental stress during EVAR procedures. In this small study, we suggest using mental stress, as evaluated with HRV, and multidisciplinary team performance, as evaluated with ICECAP, to assess SBE effectiveness in real-case EVAR procedures. This SBE program and live ICECAP observations and electrocardiogram patches was well-accepted by scrub nurses and the entire team.
Read more
A Systematic Review of Simulation-Based Training in Vascular Surgery
Alexander Haiser, Abdullatif Aydin, Basir Kunduzi, Kamran Ahmed, Prokar Dasgupta
Guy’s, King’s and St Thomas’ School of Medical Education, King’s College London, UK.
MRC Centre for Transplantation, Guy’s Hospital, King’s College London, UK.
Department of Transplant Surgery, Guy’s and St. Thomas’ NHS Foundation Trust, London, UK.
J Surg Res. 2022 Jul 12;279:409-419
Introduction: Recent advancements in surgical technology, reduced working hours, and training opportunities exacerbated by the COVID-19 pandemic have led to an increase in simulation-based training. Furthermore, a rise in endovascular procedures has led to a requirement for high-fidelity simulators that offer comprehensive feedback. This review aims to identify vascular surgery simulation models and assess their validity and levels of effectiveness (LoE) for each model in order to successfully implement them into current training curricula.
Methods: PubMed and EMBASE were searched on January 1, 2021, for full-text English studies on vascular surgery simulators. Eligible articles were given validity ratings based on Messick’s modern concept of validity alongside an LoE score according to McGaghie’s translational outcomes.
Results: Overall 76 eligible articles validated 34 vascular surgery simulators and training courses for open and endovascular procedures. High validity ratings were achieved across studies for: content , response processes , the internal structure , relations to other variables, and consequences. Only seven studies achieved an LoE greater than 3/5. Overall, ANGIO Mentor was the most highly validated and effective simulator and was the only simulator to achieve an LoE of 5/5.
Conclusions: Simulation-based training in vascular surgery is a continuously developing field with exciting future prospects, demonstrated by the vast number of models and training courses. To effectively integrate simulation models into current vascular surgery curricula and assessments, there is a need for studies to look at trainee skill retention over a longer period of time. A more detailed discussion on cost-effectiveness is also needed.
Definition of Proficiency Level by a Virtual Simulator as a First Step Toward a Curriculum on Fundamental Skills for Endovascular Aneurysm Repair (EVAR)
Andrea Moglia , Roberta Piazza , Davide Maria Mocellin , Vincenzo Ferrari , Giuseppe Campanelli, Mauro Ferrari , Raffaella Berchiolli
EndoCAS, Center for Computer-Assisted Surgery, Department of Translational Research and New Surgical and Medical Technologies, University of Pisa, Pisa, Italy. Electronic address: andrea.moglia@endocas.org.
EndoCAS, Center for Computer-Assisted Surgery, Department of Translational Research and New Surgical and Medical Technologies, University of Pisa, Pisa, Italy; Department of Information Engineering, University of Pisa, Pisa, Italy.
Vascular Surgery Unit, Department of Translational Research and of New Surgical and Medical Technologies, University of Pisa, Pisa, Italy.
Department of Law, University of Pisa, Pisa, Italy.
EndoCAS, Center for Computer-Assisted Surgery, Department of Translational Research and New Surgical and Medical Technologies, University of Pisa, Pisa, Italy; Vascular Surgery Unit, Department of Translational Research and of New Surgical and Medical Technologies, University of Pisa, Pisa, Italy.
J Surg Educ. 2021 Jan-Feb;78(1):302-303
Objective: At present, there is no proficiency-based curriculum for endovascular treatment of aortic aneurysm repair (EVAR) using virtual reality (VR) surgical simulators, whereas such curricula are available for the treatment of iliac and/or superficial femoral artery disease. The purpose of this work was to compute proficiency, defined by a benchmark level determined by the performance of experts, using a commercial VR simulator as a first step of a curriculum on EVAR.
Materials and methods: Expert endovascular surgeons (with more than 150 EVAR cases as first operators) from 12 major Italian centers completed three cases of EVAR of increasing difficulty level 3 times each, using the Angio Mentor simulator (by Simbionix) and Gore devices. Proficiency level was based on performance of expert surgeons, as assessed by metrics from a VR simulator.
Results: The participating surgeons had a median of 20 years of experience and executed a median of 440 EVAR. For the 3 simulated cases, the following proficiency values were respectively obtained: total procedure time: 22 minutes 32 seconds, 23 minutes 05 seconds, and 20 minutes 32 seconds; total amount of contrast injected: 85.16 mL, 89.97 mL, and 98.01 mL total fluoroscopy time: 10 minutes 39 seconds, 12 minutes 22 seconds, and 10 minutes 17 seconds; time to contralateral gate cannulation: 5 minutes 51 seconds, 7 minutes 09 seconds, and 3 minutes 32 seconds.
Conclusions: We computed proficiency levels for 3 simulated cases of EVAR using a VR simulator. Our next step is to determine whether surgical residents can reach this level. Translational research will then be required to assess the impact of such training on real patients.
International implementation of PROSPECT in daily practice
PROficiency based StePwise Endovascular Curricular Training
Gilles Soenens1 ∙ Jonathan Lawaetz2,3 ∙
Anne-Sophie Bamelis1 ∙ Leizl J Nayahangan2 ∙
Lars Konge2 ∙ Jonas Eiberg2,3 ∙ Isabelle Van Herzeele1
1 Department of Thoracic and Vascular Surgery, Ghent University Hospital, Belgium
2 Copenhagen Academy for Medical Education and Simulation (CAMES), Copenhagen, Denmark
3 Department of Vascular Surgery, Rigshospitalet, Copenhagen, Denmark
Presented as a poster at Paris Vascular Insights meeting, October 2021
Simulation reduces navigational errors in cerebral angiography training
Zaika, O., Boulton, M., Eagleson, R. et al
Background: Simulation-based medical education (SBME) is growing as a powerful aid in delivering proficient skills training in many specialties. Cerebral angiography (CA), a spatially and navigationally challenging endovascular procedure, can benefit from SBME by training targetable skills outside of the Angiosuite. In order to standardize and specify training requirements, navigational challenges and needs have to be identified. Furthermore, to enable successful adoption of these strategies, simulation adoption barriers, such as necessity of supervisory resources, must be reduced. In this study, we assessed the navigational challenges in simulated CA through a self-guided novice training program.
Methods: Novice participants (n = 14) received virtual reality (ANGIO Mentor, Simbionix) diagnostic cerebral angiography training and were tested on a right middle cerebral artery aneurysm case over 8 sessions with a reference instructional outline. The navigational trajectories for the guidewire and catheter were analyzed and rates in erroneous vessel access were analyzed. Participants were given a Mental Rotations Test (MRT) and were analyzed based on MRT performance.
Results: After 8 sessions, there was a significant (p < 0.05) reduction on navigational error prevalence. The L-SUB and L-CCA saw the biggest drop in erroneous access, whereas the R-ECA, the biggest consumer of error time, saw no changes in access frequency. Individuals with high MRT score performed much better (p < 0.05) than those with low MRT score.
Conclusions: Through self-guided simulation training, we demonstrated the navigational challenges encountered in simulated CA. To establish better assessments and standards in medical training, we can create self-guided training curricula aimed at correcting errors, enabling repetitive practice, and reducing human resource needs.
Virtual Reality Simulation Of Neuroendovascular Intervention Improves Procedure Speed In A Cohort Of Trainees
Joseph Dardick , Stephanie Allen , Aleka Scoco , Richard L. Zampolin , David J. Altschul
School of Medicine, Departments of Neurosurgery and Radiology, Montefiore Medical Center, Albert Einstein College of Medicine, Bronx, New York, United State
Surgical Neurology International • 2019 • 10(184)
Background: Realistic virtual reality (VR) simulators have greatly expanded the tools available for training surgeons and interventionalists. While this technology is effective in improving performance in many fields, it has never been evaluated for neuroendovascular procedures. This study aims to determine whether VR is an effective tool for improving neuroendovascular skill among trainees.
Methods: Trainees performed two VR revascularizations of a right-sided middle cerebral artery (MCA) thrombosis and their times to procedural benchmarks (time to enter internal carotid artery [ICA], traverse clot, and complete procedure) were compared. To determine whether the improvement was case specific, trainees with less procedural exposure were timed during VR left-sided ICA (LICA) aneurysm coiling before or after performing MCA thrombectomy simulations. To determine the value of observing simulations, medical students were timed during the right MCA revascularization simulations after watching other VR procedures.
Results: Trainees significantly improved their time to every procedural benchmark during their second MCA revascularization (mean decrease = 1.08, 1.57, and 2.24 min; P = 0.0072, 0.0466, and 0.0230). In addition, time required to access the LICA during aneurysm coiling was shortened by 0.77 min for each previous VR right MCA revascularization performed (P = 0.0176; r2 = 0.71). Finally, medical students’ MCA revascularization simulation times improved by 0.87 min for each prior simulation viewed (P < 0.0221; r2 = 0.96).
Conclusion: Both performance and viewing of simulated procedures produced significant decreases in time to reach neuroendovascular procedural benchmarks. These data show that VR simulation is a valuable tool for improving trainee skill in neuroendovascular procedures.
The Role of Simulation in Boosting the Learning Curve in EVAR Procedures
Vento V, Cercenelli L, Mascoli C., Gallitto E., Ancetti S., Faggioli G., Freyrie A., Marcelli E., Gargiulo M., Stella A.
Journal of Surgical Education (2018) , 75(2) Mar-Apr 2018
OBJECTIVE: Simulation may be a useful tool for training in endovascular procedures. The aim of this study was to evaluate the effect of endovascular repair of abdominal aortic aneurysms (EVAR) simulation in boosting trainees’ learning curve.
DESIGN: Ten vascular surgery residents were recruited and divided in 2 groups (Trainee Group and Control group). At a first session (t0), each resident performed 2 simulated EVAR procedures using an endovascular simulator. After 2 weeks, each participant simulated other 2 EVAR procedures in a final session (t1). In the period between t0 and t1, each resident in the Trainee Group performed 6 simulated EVAR procedures, whereas the Control Group did not perform any other simulation. Both quantitative and qualitative performance evaluations were performed at t0 and t1. Quantitative evaluation from simulator metrics included total procedural time (TP), total fluoroscopy time (TF), time for contralateral gate cannulation (TG), and contrast medium volume (CM) injected. Qualitative evaluation was based on a Likert scale used to calculate a total performance score referred to skills involving major EVAR procedural steps.
RESULTS: All residents in the Trainee Group significantly reduced TP (48 ± 12 vs 32 ± 8 minutes, t0 vs t1, p < 0.05), TF (18 ± 7 vs 11 ± 6 minutes, p < 0.05), and CM used over time (121 ± 37 vs 85 ± 26ml, p < 0.05), but not TG (5 ± 5 vs 3 ± 4 minutes, p = 0.284). In the Control Group metrics did not change significantly in any field (TP = 55 ± 11 vs 46 ± 10 minutes; TF = 25 ± 9 vs 21 ± 4 minutes; CM = 132 ± 51 vs 102 ± 42ml; TG = 6 ± 4 vs 8 ± 5 minutes, all p > 0.05). The average Trainee Group qualitative total performance score improved significantly (p < 0.05) after rehearsal sessions when compared with the Control Group.
CONCLUSION: Simulation is an effective method to improve competence of vascular surgery residents with EVAR procedures.
Copyright © 2018 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.
Endovascular Training Using a Simulation Based Curriculum is Less Expensive than Training in the Hybrid Angiosuite
Initial Results of an Original Program of Simulation Training for Endovascular Procedures
Georg ., Chenesseau B., Girsowicz E, Lejay A., Thaveau F., Papillon J., Lee J., Chakfe N.
USP, CHU de Strasbourg, Strasbourg, France
Ann Vasc Surg. 2017 Jan;38:e8-e9.
OBJECTIVES: The practice of simulation is currently swiftly developing. It makes it possible for students to carry out procedures in a controlled environ-ment without compromising the safety of patients. Endovascular surgery (EV) is ideal for practice on simulators and many models are currently available. We evaluated the first results of our program in a group of six residents beginning their training in vascular surgery.
MATERIALS AND METHODS: Our EV simulation program results from a collaborative project with Stanford University (California, USA) which is based on an evaluation of the various sequences composing an EV procedure according to a grid of evaluation comprising approximately 40 items. The procedures are carried out in a realistic environment reproducing a surgical unit. The simulator used was the ANGIO Mentor Suite (Simbionix) equipped with various modules (iliac, superficial femoral, EVAR). Each procedure was filmed using several cameras in order to evaluate the posture and the gestural of the trainees. After each procedure a “debriefing” was carried out so as to reconsider the sequences having generated difficulties and to reinforce the teaching impact. Each resident could take part in eight 2 hours sessions and was evaluated on procedures of increasing difficulty adapted to his skills. The procedures included diagnostic angiography (DA), iliac angioplasty (IA) and superficial femoral (SFA) angioplasty by crossover. At the end of each session, the residents evaluated the teaching content using a questionnaire.
RESULTS: Six residents took part in the program. None of them had preliminary experience in EV surgery. The rate of participation in the sessions was 77% (62-100%). The average progression on a scale from 0 to 100 was 24 (9-46) for DA, 35 (30-42) for IA and 21 (5-42) for the SFA. The global evaluation of the sessions by the residents showed an average score of 4.6/5.
CONCLUSION: The regular practice of simulation according to a standardized and evaluated program improves the performance of the residents on EV procedures. The feeling of the residents was excellent and they were eager to be able to take advantages of simulation sessions. EV training by simulation has a place to train the residents in vascular surgery.
A Proficiency Based Stepwise Endovascular Curricular Training (PROSPECT) Program Enhances Operative Performance in Real Life: A Randomised Controlled Trial
This study was presented at the prize session at the European Society for Vascular Surgery annual meeting, Copenhagen, September 28–30, 2016.
Eur J Vasc Endovasc Surg (2017) 54, 387-396
Maertens a,* , R. Aggarwalb,c , N. Moreels a , F. Vermassen a , I. Van Herzeele a
a Department of Thoracic and Vascular Surgery, Ghent University Hospital, Ghent, Belgium
b Department of Surgery, Faculty of Medicine, McGill University, Montreal, Canada
c Steinberg Centre for Simulation and Interactive Learning, Faculty of Medicine, McGill University, Montreal, Canada
OBJECTIVES: Healthcare evolution requires optimisation of surgical training to provide safe patient care. Operating room performance after completion of proficiency based training in vascular surgery has not been investigated.
DESIGN: A randomised controlled trial evaluated the impact of a Proficiency based Stepwise Endovascular Curricular Training program (PROSPECT) on the acquisition of endovascular skills and the transferability of these skills to real life interventions. Materials: All subjects performed two endovascular interventions treating patients with symptomatic iliac and/or superficial femoral artery stenosis under supervision. Primary outcomes were technical performances (Global Rating Scale [GRS]; Examiner Checklist), operative metrics, and patient outcomes, adjusted for case difficulty and trainee experience. Secondary outcomes included knowledge and technical performance after 6 weeks and 3 months.
METHODS: Thirty-two general surgical trainees were randomised into three groups. Besides traditional training, the first group (n ¼ 11) received e-learning and simulation training (PROSPECT), the second group (n ¼ 10) only had access to e-learning, while controls (n ¼ 11) did not receive supplementary training.
RESULTS: Twenty-nine trainees (3 dropouts) performed 58 procedures. Trainees who completed PROSPECT showed superior technical performance (GRS 39.36 2.05; Checklist 63.51 3.18) in real life with significantly fewer supervisor takeovers compared with trainees receiving e-learning alone (GRS 28.42 2.15; p ¼ .001; Checklist 53.63 3.34; p ¼ .027) or traditional education (GRS 23.09 2.18; p ¼ .001; Checklist 38.72 3.38; p ¼ .001). Supervisors felt more confident in allowing PROSPECT trained physicians to perform basic (p ¼ .006) and complex (p ¼ .003) procedures. No differences were detected in procedural parameters (such as fluoroscopy time, DAP, procedure time, etc.) or complications. Proficiency levels were maintained up to 3 months.
CONCLUSIONS: A structured, stepwise, proficiency based endovascular curriculum including e-learning and simulation based training should be integrated early into training programs to enhance trainee performance.
Simulator-Based Angiography and Endovascular Neurosurgery Curriculum: A Longitudinal Evaluation of Performance Following Simulator-Based Angiography Training
Pannell JS1, Santiago-Dieppa DR1, Wali AR1, Hirshman BR1, Steinberg JA1, Cheung VJ1, Oveisi D2, Hallstrom J3, Khalessi AA1.
August 29, 2016
Author information
- 1Department of Neurosurgery, University of California, San Diego.
- 2Department of Internal Medicine, University of California, Los Angeles.
- 3Neuroradiology, University of New Mexico.
Endovascular aneurysm repair simulation can lead to decreased fluoroscopy time and accurately delineate the proximal seal zone
Ann H Kim1, Daniel E Kendrick 1, Pamela A Moorehead 2, Anil Nagavalli 2, Claire P Miller 2, Nathaniel T Liu 1, John C Wang 1, Vikram S Kashyap 1
1Division of Vascular Surgery, Harrington Heart & Vascular Institute, University Hospitals Case Medical Center, Cleveland, Ohio.
2Department of Physiology and Biophysics, Case Western Reserve University, Cleveland, Ohio.
J Vasc Surg, 2016 Jul;64(1):251-8
Background: The use of simulators for endovascular aneurysm repair (EVAR) is not widespread. We examined whether simulation could improve procedural variables, including operative time and optimizing proximal seal. For the latter, we compared suprarenal vs infrarenal fixation endografts, right femoral vs left femoral main body access, and increasing angulation of the proximal aortic neck.
Methods: Computed tomography angiography was obtained from 18 patients who underwent EVAR at a single institution. Patient cases were uploaded to the ANGIO Mentor endovascular simulator (Simbionix, Cleveland, Ohio) allowing for three-dimensional reconstruction and adapted for simulation with suprarenal fixation (Endurant II; Medtronic Inc, Minneapolis, Minn) and infrarenal fixation (C3; W. L. Gore & Associates Inc, Newark, Del) deployment systems. Three EVAR novices and three experienced surgeons performed 18 cases from each side with each device in randomized order (n = 72 simulations/participant). The cases were stratified into three groups according to the degree of infrarenal angulation: 0° to 20°, 21° to 40°, and 41° to 66°. Statistical analysis used paired t-test and one-way analysis of variance.
Results: Mean fluoroscopy time for participants decreased by 48.6% (P < .0001), and total procedure time decreased by 33.8% (P < .0001) when initial cases were compared with final cases. When stent deployment accuracy was evaluated across all cases, seal zone coverage in highly angulated aortic necks was significantly decreased. The infrarenal device resulted in mean aortic neck zone coverage of 91.9%, 89.4%, and 75.4% (P < .0001 by one-way analysis of variance), whereas the suprarenal device yielded 92.9%, 88.7%, and 71.5% (P < .0001) for the 0° to 20°, 21° to 40°, and 41° to 66° cases, respectively. Suprarenal fixation did not increase seal zone coverage. The side of femoral access for the main body did not influence proximal seal zone coverage regardless of infrarenal angulation.
Conclusions: Simulation of EVAR leads to decreased fluoroscopy times for novice and experienced operators. Side of femoral access did not affect precision of proximal endograft landing. The angulated aortic neck leads to decreased proximal seal zone coverage regardless of infrarenal or suprarenal fixation devices.
Diagnostic coronary angiography: initial results of a simulation program
David B. Casey, David Stewart, Mladen I. Vidovich
Division of Cardiology, University of Illinois at Chicago
Cardiovascular Revascularization Medicine, March 2016
BACKGROUND: The use of simulator-based teaching in cardiology has unfortunately lagged behind other procedural specialties. This study investigates the utility of a simulator-based training program for fellows in cardiovascular disease with no prior experience in diagnostic coronary angiography.
METHODS: First-year cardiology fellows at University of Illinois-Chicago (UIC) using ANGIO Mentor™ simulators completed benchmark cases requiring basic coronary engagement. Subsequently, benchmark cases were completed one day later and at 9 months following 2-3 months of training in the cardiac catheterization lab. In addition, 1st year cardiology fellows were compared to 3rd year fellows. Objective measures assessed from benchmark cases were total procedural time, total contrast used, and total fluoroscopy time.
RESULTS: All 1st year fellows improved their total time to complete the benchmark case from initial to second attempt one day later (14:56 on Day 1, 8:30 on Day 2, P=0.03). Total contrast used (60 mL on Day 1, 39 mL on Day 2, P=0.11) and total fluoroscopic time (6:30 on Day 1 and 4:26 on Day 2, P=0.16) also both decreased. Overall procedure time and contrast use was similar among 1st and 3rd year fellows after simulation training. Decreases in procedure and fluoroscopy time were maintained in 1st year fellows after 2-3 months of training.
CONCLUSIONS: Fellows displayed technical and procedural improvement at diagnostic coronary angiography in a short period of time and in a safe, patient free environment. In this study, a computer-based simulator was successfully incorporated into a first year cardiovascular fellowship curriculum and represents a contemporary means to provide the fellow increased procedural training without added risk to the patient.
The model for Fundamentals of Endovascular Surgery (FEVS) successfully defines the competent endovascular surgeon
Cassidy Duran, MD, Sean Estrada, PhD, Marcia O’Malley, PhD, Malachi G. Sheahan, MD, Murray L. Shames, MD, Jason T. Lee, MD, Jean Bismuth, MD
University of South Florida, Tampa, Fla
Journal of Vascular Surgery 2015 Dec;62(6):1660-1666.
OBJECTIVE: Fundamental skills testing is now required for certification in general surgery. No model for assessing fundamental endovascular skills exists. Our objective was to develop a model that tests the fundamental endovascular skills and differentiates competent from noncompetent performance.
METHODS: The Fundamentals of Endovascular Surgery model was developed in silicon and virtual-reality versions. Twenty individuals (with a range of experience) performed four tasks on each model in three separate sessions. Tasks on the silicon model were performed under fluoroscopic guidance, and electromagnetic tracking captured motion metrics for catheter tip position. Image processing captured tool tip position and motion on the virtual model. Performance was evaluated using a global rating scale, blinded video assessment of error metrics, and catheter tip movement and position. Motion analysis was based on derivations of speed and position that define proficiency of movement (spectral arc length, duration of submovement, and number of submovements).
RESULTS: Performance was significantly different between competent and noncompetent interventionalists for the three performance measures of motion metrics, error metrics, and global rating scale. The mean error metric score was 6.83 for noncompetent individuals and 2.51 for the competent group (P < .0001). Median global rating scores were 2.25 for the noncompetent group and 4.75 for the competent users (P < .0001).
CONCLUSIONS: The Fundamentals of Endovascular Surgery model successfully differentiates competent and noncompetent performance of fundamental endovascular skills based on a series of objective performance measures. This model could serve as a platform for skills testing for all trainees.
Validation of a Novel Virtual Basic Skills Simulation Model
Murray L. Shames, Johnathon V. DallaRosa, Michael T. Brannick
University of South Florida, Tampa, Fla
Journal of Vascular Surgery 2015 Jun;61(6):99S
OBJECTIVES: This study determined the construct validity of a novel basic endovascular skills challenge on a virtual reality simulator.
METHODS: Subjects completed a basic endovascular skills challenge on the Simbionix ANGIO Mentor. The subjects were divided into three groups based on endovascular experience: novice (medical students and postgraduate year 1 surgery residents), intermediate (vascular surgery residents postgraduate year 2-7), and expert (vascular surgery faculty). Each task requires advancing a wire, catheter, and sheath to predetermined positions in a virtual vascular tree. The challenge is divided into five cases. Case 1 contains five tasks. Each subsequent task is more complicated than the previous task. Time to completion and points (generated by the simulator) were compared for case 1 (tasks 1-5). Time and points were compared with each other to determine construct validity.
RESULTS: Twenty-two subjects (11 novice, 6 intermediate, and 5 expert) completed the simulation. One-way analysis of variance was computed for time to case completion with the group as the independent variable. The result was statistically significant (P < .05). The results of the Tukey least significant difference post hoc test showed that the novice group was significantly different from the other two groups, which did not differ from one another. One-way analysis of variance was also computed for points accrued for case completion with the same groups. The result was statistically significant (P < .05). Post hoc tests were also computed for the groups, and the results were similar to those for seconds to completion. The novice group earned fewer points than the other two groups, which did not differ from one another. A scatterplot of seconds by points for the case reveals a strong but nonlinear association that is to be expected from the scoring method that assigns points by time.
CONCLUSIONS: A novel endovascular skills challenge on the Simbionix ANGIO Mentor is able to differentiate novice from intermediate and expert operators.
Development of a PROficiency-Based StePwise Endovascular Curricular Training (PROSPECT) Program
Heidi Maertens, MD1, Rajesh Aggrawal, MD, PhD2, Liesbeth Desender1, MD, Frank Vermassen1, MD, PhD, Isabelle Van Herzeele1, MD, PhD.
1 Department of Thoracic and Vascular Surgery, Ghent University Hospital, Ghent, Belgium
2 Department of Surgery, McGill University, Montreal, Canada
Journal of Surgical Education, Published Online: August 11, 2015
DESIGN: A PROficiency-based StePwise Endovascular Curricular Training (PROSPECT) program was developed, consisting of e-learning and hands-on simulation modules, focusing on iliac and superficial femoral artery atherosclerotic disease. Construct validity was investigated. Performances were assessed using multiple-choice questionnaires, valid simulation parameters, global rating scorings, and examiner checklists. Feasibility was assessed by passage of 2 final-year medical students through this PROSPECT program.
SETTING: Ghent University Hospital, a tertiary clinical care and academic center in Belgium with general surgery residency program.
PARTICIPANTS: Senior-year medical students were recruited at Ghent University Hospital. Vascular surgeons were invited to participate during conferences and meetings if they had performed at least 100 endovascular procedures as the primary operator during the last 2 years.
RESULTS: Overall, 29 medical students and 20 vascular surgeons participated. Vascular surgeons obtained higher multiple-choice questionnaire scores (median: 24.5-22.0 vs. 15.0-12.0; p < 0.001). Students took significantly longer to treat any iliac or femoral artery stenosis (3.3-14.8 vs. 5.8-30.1 min; p = 0.001-0.04), whereas in more complex cases, fluoroscopy time was significantly higher in students (8.3 vs. 21.3 min; p = 0.002; 7.3 vs. 13.1 min; p = 0.03). In all cases, vascular surgeons scored higher on global rating scorings (51.0-42.0 vs. 29.5-18.0; p < 0.001) and examiner checklist (81.5-75.0 vs. 54.5-43.0; p < 0.001). Hence, proficiency levels based on median expert scores could be determined. There were 2 students who completed the program and passed for each step within a 3-month period during their internships.
CONCLUSIONS: A feasible and construct validated surgical program to train cognitive, technical, and nontechnical endovascular skills was developed. A structured, stepwise, proficiency-based valid endovascular program to train cognitive, technical, and human factor skills has been developed and proven to be feasible. A randomized controlled trial has been initiated to investigate its effect on performances in real life, patient outcomes, and cost-effectiveness.
Diagnostic angiography skill acquisition with a secondary curve catheter: phase 2 of a curriculum-based endovascular simulation program
Spiotta AM1, Kellogg RT1, Vargas J1, Chaudry MI2, Turk AS2, Turner RD1.
1 Division of Neurosurgery, Department of Neurosciences, Medical University of South Carolina, Charleston, South Carolina, USA.
2 Department of Neuroradiology, Medical University of South Carolina, Charleston, South Carolina, USA
J Neurointerv Surg. 2014 Sep 3.
BACKGROUND: We have previously reported the efficacy of a simulator-based training paradigm for residents in neurosurgery with little or no prior experience in diagnostic cerebral angiography with straightfoward arch anatomy. This study investigates the utility of a simulation-based training curriculum for the acquisition of skills employing a secondary curve catheter to navigate more complex arch anatomy.
METHODS: Residents at the Medical University of South Carolina (MUSC) with moderate exposure to diagnostic angiography enrolled into a standardized Institutional Review Board-approved training protocol using SimSuite Compass and Simbionix simulators. The task involved (in order) forming the Simmons catheter in the left subclavian artery and then selecting the brachiocephalic, left common carotid and left vertebral arteries.
RESULTS: All participants improved their total time to complete the task over the course from the first to last trial. Each milestone within the overall task also demonstrated an improvement across trials for each participant. Following the hands-on experience, participants’ rating of their knowledge of arch anatomy and vessel selection technique improved to that between competence and high competence (values of 3.3±0.49 (p<0.005) and 3.1±0.38 (p<0.01), respectively). Comfort with use of the Simmons catheter improved to a value of 2.9±0.38 (p<0.001), between an experienced learner and competence. Participants rated the usefulness of the training environment as very high (4.1±0.90 out of maximum 5).
CONCLUSIONS: Residents became more proficient at vessel selection in a type II and bovine arch over a relatively compressed time period, with both objective and subjective data demonstrating acquisition of skill sets and increased confidence.
Endovascular Simulation Leads to Efficiency and Competence in TEVAR Procedures
Andre F. Gosling, Anil Nagavalli, Daniel Kendrick, Vikram S. Kashyap, John C. Wang
Vascular Surgery, University Hospitals Case Medical Center, Cleveland, Ohio
Journal of Vascular Surgery Volume 59, Issue 6, Supplement, Page 13S, June 2014
OBJECTIVES: We assessed the effects of thoracic endovascular aortic repair (TEVAR) rehearsal with an endovascular surgical simulator in surgical trainees at different levels in training
METHODS: Twelve trainees in three cohorts (student, postgraduate year [PGY]1-3, and PGY 4-7) were oriented to using the Simbionix AngioMentor simulator over four sessions. Likert scale qualitative analysis evaluated participant proficiency. Analysis of data included one-way analysis of variance and paired t-tests.
RESULTS: All groups had reduction of total procedure time (mean 537 ± 148 seconds vs 269 ± 66 seconds, first session vs fourth; confidence interval [CI], 195-341 seconds; P < .05) and fluoroscopy time (201 ± 74 vs 110 ± 37 seconds; CI, 51-132 seconds; P < .05) with case progression. Procedure time significantly decreased in students (551 ± 84 vs 313 ± 65 seconds; CI 189-287 seconds; P < .05) and PGY 1-3 (591 ± 149 vs 264 ± 29 seconds; CI, 113-541 seconds; P < .02) had a significant decrease in procedure time. Fluoroscopy times were brief overall between individuals and groups and did not change significantly with case progression. Participants acquired proficiency after a few runs in almost every step of the procedure. Endograft sizing appeared to be the most challenging task by qualitative analysis. PGY 4-7 trainees had higher technical scores, but this was not statistically significant.
Fig 1. Comparison among groups in TEVAR total procedure time
Conclusions: Practice on endovascular surgical simulators can reduce overall procedure and fluoroscopy time, independent of trainee skill level or experience. Endovascular simulators can be effective tools in residency training and maintenance of proficiency. Further studies are needed to compare simulator performance with outcomes in real cases.
Realism, criterion validity, and training capability of simulated diagnostic cerebral angiography.Nguyen N1, Eagleson R1, Boulton M2, de Ribaupierre S1.
1 Department of Electrical and Computer Engineering, Western University, London, ON, Canada.
2 Clinical Neurological Sciences, Schulich School of Medicine and Dentistry, Western University, London, ON, Canada.
Stud Health Technol Inform. 2014;196:297-303.
Computer-based simulation is increasingly used in medical education for training, assessment, credentialing, and practice. Compared to medical specialties such as anesthesiology and general surgery, the adoption of simulation for neurointerventional training has been slow. This may be due to the limited number of neurointerventional simulators available and the lack of research assessing their validity and training capability. The objective of this study was to assess the realism, validity, and training capability of computer-based simulation for diagnostic cerebral angiography using a commercially available simulator called the ANGIO Mentor Express.
The following abstract was presented at the Royal College Simulation Summit , November 8 – 9, 2013, Vancouver, Canada
Criterion and face validity of the ANGIO Mentor for diagnostic cerebral angiography
Ngan Nguyen, PhD1, Roy Eagleson, PhD1, Mel Boulton, PhD, MD2, Sandrine deRibaupierre, MD1,2
1 Department of Electrical and Computer Engineering,Western University, London, ON, Canada
2 Department of Clinical Neurological Sciences, Western University, London, ON, Canada
The objective of this study was to assess the face and criterion validity of a computer-based simulation for diagnostic cerebral angiography using the ANGIO Mentor Express. Participants were divided into two groups: experts (experienced interventional physicians) and novices (residents and fellows). Face validity was addressed by asking experts to rate, on a 5-point Likert scale, the appropriateness of the simulated content as a teaching and training tool. Criterion validity was established by comparing the simulation performance of experts vs. novices. After completing a step-by-step tutorial to become acquainted with the relevant technical features involved in performing a simulated cerebral angiography, all participants practiced performing an angiography of the left internal carotid artery. Subsequently, they completed a simulated angiography of the right middle cerebral artery. The procedure time, fluoroscopy time, amount of contrast, number of fluoroscopic images, and number of roadmaps utilized when performing the right middle cerebral artery were recorded. These allowed us to compute objective measures of performance. Upon completion of the two simulated cases, experts were asked to rate the appropriateness of the simulated content. Experts outperformed novices in nearly all performance variables, but significant differences were found for fluoroscopy time and amount of contrast utilized, p<0.05. Experts reported that the ANGIO Mentor provided content appropriate to the angiography procedure (mean=4.85) and that it is useful as a teaching and training tool (mean=4.71). Preliminary results revealed that the ANGIO Mentor has appropriate face and criterion validity, providing support for the ANGIO Mentor’s use as a tool for teaching diagnostic cerebral angiography.
Two Centre Evaluation of the ANGIO Mentor Electrophysiology Simulator
- Ullah1, R. Marazzi2, R. Hunter1, M. Dhinoja1, S. Sporton1, R. De Ponti2, R. Schilling1
1 Department of Cardiology, St Bartholomew’ s Hospital, Barts Health Trust, London, UK; 2Department of Clinical and Experimental Medicine University of Insubria, Varese, Italy.
Presented as a poster at HRS meeting, 2013
Objective Simulator-Based Evaluation of Carotid Artery Stenting Proficiency (from Assessment of Operator Performance by theCarotid Stenting Simulator Study [ASSESS])
Giora Weisz, MD, Nathaniel R. Smilowitz, MD, Helen Parise, ScD, Jacques Devaud,
Issam Moussa, MD, Stephen Ramee, MD, Mark Reisman, MD, Christopher J. White, MD,
and William A. Gray, MD
American Journal of Cardiology, April 2013
Studies have suggested that operator proficiency has a substantial effect on complication rates and procedural outcomes. Endovascular simulators have been used for training and have been proposed as an alternative to the conventional assessment of skills. The present study sought to validate simulation as an objective method for proficiency evaluation in carotid artery stenting. Interventional cardiologists classified as novice, intermediate, or experienced practitioners performed 3 simulated, interactive carotid stenting cases on an Angio Mentor endovascular simulator. An automated algorithm scored the participants according to the technical performance, medical management, and angiographic results. A total of 33 interventional cardiologists (8 novices, 15 intermediates, and 10 experts) completed 82 simulated procedures. The composite scores for the case simulations varied significantly by operator experience, with better scores for the more experienced groups (p <0.0001). The metrics that discriminated between operator experience groups included fluoroscopy time, crossing the carotid lesion with devices other than a 0.014-in. wire before filter deployment, and incomplete coverage of the lesion by the stent. In conclusion, the results of the present study validate that a simulator with an automated scoring system is able to discriminate between levels of operator proficiency for carotid artery stenting. Simulator based performance assessment could have a role in initial and ongoing proficiency evaluations and credentialing of interventional operators of high-risk endovascular procedures.
Simulated diagnostic cerebral angiography in neurosurgical training: a pilot program
Spiotta AM, Rasmussen PA, Masaryk TJ, Benzel EC, Schlenk R.
Department of Neurological Surgery, Cleveland Clinic, Cleveland, Ohio, USA.
J Neurointerv Surg. 2012, May 10.
INTRODUCTION: Surgical simulation provides a zero-risk setting in which technical skills can be obtained through repetition. The feasibility and utility of simulated diagnostic cerebral angiography among neurosurgical residents and fellows was studied using an endovascular biplane angiography simulator.
METHODS: Ten neurosurgical residents and four endovascular neurosurgery fellows were recruited into a standardized training protocol consisting of a didactic, demonstration and hands-on learning environment using the Simbionix simulator. Participants were instructed to catheterize the right internal carotid artery, left internal carotid artery and left vertebral artery. The task was repeated five times.
RESULTS: All participants demonstrated improvement over the five trials. Residents performed actions that were perceived as potentially dangerous (n=8) while fellows performed the procedure with superior technique. Residents performed the task with an initial total procedure and fluoroscopy time of 6.6±4.3 min and 4.9±3.7 min, respectively, and improved on the fifth trial to 3.4±1.3 min (p=0.03) and 2.3±0.78 min (p=0.004), respectively. Residents approximated the efficiency of fellows for the third and fourth trial.
CONCLUSIONS: Incorporating an endovascular simulator is feasible for training purposes in a neurosurgical residency program. This study provides objective documentation of the facilitation of technical angiography skill acquisition by the use of simulation technology.
The following study by Stanford researchers reveal results from endovascular skills training for surgical residents and was presented at the at the 66th Vascular Annual Meeting presented by the Society for Vascular Surgery, June 7-9, 2012, at Gaylord National Resort & Convention Center, National Harbor, MD.
Results from endovascular skills training for surgical residents
Jason T. Lee
Stanford University’s Goodman Simulation Center in the Department of Surgery in Palo Alto, CA,
Training with simulation versus operative room attendance.
Desender LM, Van Herzeele I, Aggarwal R, Vermassen FE, Cheshire NJ.
Department of Thoracic and Vascular Surgery, University Hospital Ghent, Ghent, Belgium.
J Cardiovasc Surg (Torino). 2011 Feb;52(1):17-37.
Reduced training times, increasing complexity of endovascular and open vascular interventions and concerns for patient’s safety have necessitated a modernization in surgical training. A more strategic approach is required to facilitate the acquisition of surgical skills outside the operating room and to minimize the risks to patients as surgeons develop their technical expertise. Virtual reality simulation has been proposed as a means to train and objectively assess technical endovascular performance without risks to patient safety. This article reviews the evidence and the limitations for this adjunctive tool, the implementation in current training programs and future applications to maintain the highest standards of care for treatment of vascular disease.
Simulation in Neurosurgical Residency Training: A New Paradigm
Alejandro M. Spiotta, MD Richard P. Schlenk, MD
The Cleveland Clinic Foundation, Cleveland, Ohio, USA
The Congress of Neurological Surgeons(CNS) Quarterly 2010 page 18-20
In the era of duty hour restrictions and increasing medico-legal pressures, surgical simulation offers a viable alternative to bridge the gap in experience and knowledge of residents. The value of simulation for the re-creation of invasive procedures has a rich history in cadaveric dissection and animal experimentation. While these exposures provide a valuable experience and do not require sophisticated technological support, cadavers and laboratory animals are a scarce and expensive resource. Three years ago, we established a ‘fundamental skills laboratory’ funded by educational grant support. The course was run by our faculty and senior level residents and provided an educational experience to the junior level residents. The course involved intensive hands-on exercises designed to familiarize the trainees with fundamental bedside procedures and operative skills such as external ventricular drain and intracranial pressure bolt placement, suturing, drilling and turning a craniotomy flap. This course was extremely successful and was the forerunner to the SNS Boot Camp that will be offered for its second consecutive year in July 2011 for all incoming post-graduate graduate year 1 (PGY1) neurosurgery residents.
Virtual Reality Simulation in the Endovascular Field
Aggarwal Rajesh , Herzeele Isabelle Van
European Virtual Reality Endovascular Research Team (EVEREST)
US Cardiology, 2008;5(1):41-5
The last decade has witnessed exponential growth in the field of endovascular interventions, although only in the last few years has there been a widespread interest in the carotid artery stent (CAS). Endovascular physicians with different medical backgrounds such as interventional cardiologists, radiologists, and vascular surgeons all recognize the importance of this changing technology.1This procedure is almost unique, as the risks to the patient (stroke and death) as a result of the physician’s learning curve are unacceptably high. This has been summarized by editorials written after the publication of the Carotid and Vertebral Artery Angioplasty Study (CAVATAS)2–4 and the Endarterectomy versus Angioplasty in Patients with Symptomatic Severe Carotid Stenosis trial (EVA-3S).5–7 Recent publications of the rates of medical errors and adverse events within healthcare8 have drawn the spotlight toward methods of establishing credentials for physicians preparing to perform complex procedures. In order to improve patient safety, operators must have the appropriate cognitive and technical skills and experience of CAS. Furthermore, physicians should have previously achieved a high level of proficiency in other catheter-based interventions and completed dedicated training in CAS.9
The Utility of Endovascular Simulation to Improve Technical Performance and Stimulate Continued Interest of Preclinical Medical Students in Vascular Surgery
Jason T. Lee , Mary Qiu , Mediget Teshome, Shyam S. Raghavan, Maureen M. Tedesco, and Ronald L. Dalman
Division of Vascular Surgery, Stanford University School of Medicine, Stanford, California
Journal of Surgical Education Volume 66, Issue 6, November-December 2009, Pages 367-373 Available online 30 January 2010.
Presented at the Association for Program Directors in Surgery/Association for Program Directors in Vascular Surgery Meeting, April 30, 2009, Salt Lake City, UT.
OBJECTIVES: New training paradigms in vascular surgery allow for early specialization out of medical school. Surgical simulation has emerged as an educational tool for trainees to practice procedures in a controlled environment allowing interested medical students to perform procedures without compromising patient safety. The purpose of this study is to assess the ability of a simulation-based curriculum to improve the technical performance and interest level of medical students in vascular surgery.
DESIGN: Prospective observational cohort study of medical student performance.
SETTING: Academic medical center.
PARTICIPANTS: Forty-one medical students (23 first year, 15 second year, 3 other) enrolled in a vascular surgery elective course. Students completed a survey of their interests and performed a renal stent procedure on an endovascular simulator (pretest). The curriculum consisted of didactic teaching and weekly mentored simulator sessions and concluded with a final renal stent procedure on the simulator (posttest). Objective procedural measures were determined during the pre- and posttest by the simulator, and subjective performance was graded by expert observers utilizing a structured global assessment scale. After the course, the students were surveyed as to their opinions about vascular surgery as a career option. Finally, 1 year after the course, all students were again surveyed to determine continued interest in vascular surgery.
RESULTS: The objective and subjective criteria measured on the simulator and structured global assessment scale significantly improved from pre- to posttest in terms of performer technical skill, patient safety measures, and structured global assessments. Before beginning the course, 8.5% of the students expressed high interest in vascular surgery, and after completing the course 70% were seriously considering vascular surgery as a career option (p = 0.0001). More than 95% of the students responded that endovascular simulation increased their knowledge and interest in vascular surgery. In the 1-year follow-up survey (n = 23 medical students), 35% had already entered their clinical years. Seventy percent of the students were still considering vascular surgery, while several other career options were still popular including the surgical subspecialties (70%), interventional cardiology (57%), and interventional radiology (48%). Most respondents indicated the major reasons for continued interest in vascular surgery were the ability to practice endovascular procedures on the simulator (100%) and mentorship from vascular surgery faculty (78%).
CONCLUSIONS: The use of high fidelity endovascular simulation within an introductory vascular surgery course improves medical student performance with respect to technical skill, patient safety parameters, and global performance assessment. Mentored exposure to endovascular procedures on the simulator positively impacts long term medical student attitudes towards vascular surgery. Simulator-based courses may have the potential to be an important component in the assessment and recruitment of medical students for future surgical training programs.
Experienced Endovascular Interventionalists Objectively Improve their Skills by Attending Carotid Artery Stent Training Courses
Van Herzeele I, Aggarwal R, Neequaye S, Hamady M, Cleveland T, Darzi A, Cheshire N, Gaines P.
Department of Biosurgery and Surgical Technology, Imperial College London, UK; Regional Vascular Unit, St. Mary’s Hospital, London, UK.
Eur J Vasc Endovasc Surg. 2008 May;35(5):541-50. Epub 2008 Feb 8.
OBJECTIVES: Carotid artery stenting (CAS) is an advanced endovascular intervention with a steep learning curve. Virtual reality (VR) simulation has been proposed as a means to train and objectively assess technical performance. AIM: To objectively assess psychomotor skills acquisition of experienced interventionalists attending a two-day CAS course, using a VR simulator.
METHODS: Both cognitive and technical skills of 11 interventionalists were trained in a two-day course using didactic sessions, case reviews, supervised VR simulation and live-cases. Pre- and post-course skills were assessed through performance on the same CAS procedure using metrics derived from the simulator.
RESULTS: Significant differences were noted between pre- and post-course performance for procedure (36 vs. 20min., p=0.005), X-ray (20 vs. 11min., p=0.016) and delivery-retrieval time of the embolic protection device (12 vs. 9min., p=0.007). Advancement of the guiding catheter without a leading wire occurred to a greater extent pre- versus post-course (199 vs. 152mm., p=0.050) as did spasm of the internal carotid artery (4 vs. 2, p=0.049).
CONCLUSIONS: This study has objectively proven a benefit for experienced interventionalists to attend CAS courses for skills acquisition measured by a VR simulator. These data can be used to offer participants an insight into their skills and objectively audit course efficacy.
The following abstract was presented in the prize session of the European Society for Vascular Surgery (ESVS) Annual Meeting September 20 – 23, 2007 in Madrid, Spain
Experienced Endovascular Interventionalists Objectively Improve their Skills by Attending Carotid Artery Stent Training Courses
S. Neequaye, I. Van Herzeele, R. Aggarwal, M. Hamady, A. Darzi, T. Cleveland, P. Gaines, N. Cheshire
Department of Biosurgery and Surgical Technology, Imperial College London, U.K.
OBJECTIVES: Carotid artery stenting is an advanced endovascular intervention with a steep learning curve. Its success is tempered by the need to ensure that all interventionalists possess technical proficiency prior to performing cases on patients. Virtual reality simulation has been proposed as a means to train and objectively assess technical performance without risks to patient safety. The aim of this study was, using a commercially available simulator, to objectively assess psychomotor skills acquisition of experienced interventionalists attending a two-day carotid artery stent course.
METHODS: Eleven experienced endovascular physicians from several medical disciplines with minimal experience in carotid artery stenting participated in this study. During two identical two-day courses, cognitive and technical skills were trained using didactic sessions, case reviews, supervised VR simulation and live-cases. Pre- and post-course skills were assessed through performance on the same carotid artery stent procedure using the quantitative and qualitative assessment parameters derived from the simulator. The Wilcoxon signed ranks test was used to compare performance across the two groups.
RESULTS: Significant differences were noted between pre- and post-course performance for total procedure time (medians 36 vs. 20 minutes, p=0.005), fluoroscopic time (20 vs. 11 minutes, p= 0.016) and delivery-retrieval time of the embolic protection device (12 vs. 9 minutes, p= 0.007). Advancement of the guiding catheter or guiding sheath without a leading wire occurred to a greater extent pre-course versus post-course (199 vs. 152 millimetres, p= 0.050) as did spasm of the internal carotid artery (4 vs. 2, p= 0.049).
CONCLUSIONS: This study has objectively proven a benefit for experienced interventionalists to attend carotid artery stenting courses for skills acquisition measured by a virtual reality simulator. In addition to providing participants with an insight into their skills, these data can be used to objectively audit course efficacy.
Analysis of Simulated Angiographic Procedures. Part 2: Extracting Efficiency Data from Audio and Video Recordings
Duncan JR, Kline B, Glaiberman CB.
Mallinckrodt Institute of Radiology, Washington University School of Medicine, 510 S. Kingshighway Blvd., St. Louis, MO 63110, USA.
J Vasc Interv Radiol. 2007 Apr;18(4):535-44
PURPOSE: To create and test methods of extracting efficiency data from recordings of simulated renal stent procedures.
MATERIALS AND METHODS: Task analysis was performed and used to design a standardized testing protocol. Five experienced angiographers then performed 16 renal stent simulations using the Simbionix AngioMentor angiographic simulator. Audio and video recordings of these simulations were captured from multiple vantage points. The recordings were synchronized and compiled. A series of efficiency metrics (procedure time, contrast volume, and tool use) were then extracted from the recordings. The intraobserver and interobserver variability of these individual metrics was also assessed. The metrics were converted to costs and aggregated to determine the fixed and variable costs of a procedure segment or the entire procedure.
RESULTS: Task analysis and pilot testing led to a standardized testing protocol suitable for performance assessment. Task analysis also identified seven checkpoints that divided the renal stent simulations into six segments. Efficiency metrics for these different segments were extracted from the recordings and showed excellent intra- and interobserver correlations. Analysis of the individual and aggregated efficiency metrics demonstrated large differences between segments as well as between different angiographers. These differences persisted when efficiency was expressed as either total or variable costs.
CONCLUSIONS: Task analysis facilitated both protocol development and data analysis. Efficiency metrics were readily extracted from recordings of simulated procedures. Aggregating the metrics and dividing the procedure into segments revealed potential insights that could be easily overlooked because the simulator currently does not attempt to aggregate the metrics and only provides data derived from the entire procedure. The data indicate that analysis of simulated angiographic procedures will be a powerful method of assessing performance in interventional radiology.
The Use of Interventional Cardiovascular Simulation to Evaluate Operator Performance: The Carotid Assessment of Operator Performance by the Simbionix Carotid StEnting Simulator Study (ASSESS)
Giora Weisz, Jacque Devaud, Stephen Ramee, Mark Reisman, William Gray
Cardiovascular Research Foundation, and Center for Interventional Vascular Simulation, New-York Presbyterian Hospital, Columbia University, New York, NY
Journal of the Society for Simulation in Healthcare 2007, Volume2, Issue 1
BACKGROUND: The percutaneous endovascular carotid stenting procedure has only recently been approved by the FDA, and interventional operators from multiple disciplines (Cardiology, Radiology, Vascular Surgery) are getting training in this field. The professional societies of these disciplines are looking for novel ways to assess the competency of an operator, before they grant authorization to perform this complex procedure.
Simulators have been suggested as tools to evaluate operators in specific procedures and operative techniques. The virtual reality machines can be used to capture numerous factors that translate into the success and safety of the procedure. These include selection of devices, accuracy of manual performance, reaction to interactive problems, use of adjuvant pharmacotherapy, management of hemodynamic changes, and prevention and resolution of complications. All of these can be automatically evaluated and scored by a complex software designed virtual reality simulator. The simulators that are used in Interventional Cardiology and endovascular therapy have not been validated yet as such assessment tool.
OBJECTIVE: The main objective of the ASSESS study is to validate the simulator-based metrics as a tool to discriminate between different levels of operators.
METHODS: We used the Angio-Mentor carotid Stenting simulator system (Simbionix). The simulators had been loaded with predefined cases demonstrating increasing complexity for stenting intervention.
Total of 34 operators (10 experts, 12 intermediate level, and 12 novice; based on prior reported experience) were asked each to do three cases of carotid stenting on the simulator. All the operators had the same cases that were selected to demonstrate increased complexity to judge anatomy, correct performance of the technical steps, use of wires, catheters, embolic protection device, balloons, and stents, adjuvant pharmacotherapy, and management of interactive hemodynamic response. During the procedure, multiple features of all of the above elements were automatically captured by the simulator. Each step and parameter captured had a pre-defined score. Scoring was based on penalty points for mal-performance, as pre-defined by a panel of independent experts in carotid stenting, prior to conducting the study.
RESULTS: The study has been completed, and all the data that were captured is being analyzed. Full results will be available at the time of the presentation
CONCLUSIONS: Being able to demonstrate the ability of a hi-tech, virtual reality simulator to differentiate between different levels of interventional operators, may enlighten the simulator as a potential objective “examination tool”, to evaluate the performance and competence of an interventional operator. These will justify the use of Interventional Cardiovascular Simulation for certification and credentialing.