HYST MENTOR CLINICAL VALIDATIONS
Using virtual-reality simulation to ensure basic competence in hysteroscopy
Savran MM, et al. Surg Endosc. 2018.
Surg Endosc. 2018 Oct 17. doi: 10.1007/s00464-018-6495-3. [Epub ahead of print]
BACKGROUND: Hysteroscopy is a technically challenging procedure. Specialty curricula of obstetrics and gynaecology appraise hysteroscopy for trainees but there is no present evidence-based training program that certifies the fundamental technical skills before performance on patients. The objectives of this study were to develop and gather validity evidence for a simulation-based test that can ensure basic competence in hysteroscopy.
METHODS: We used the virtual-reality simulator HystMentor™. Six experts evaluated the feasibility and clinical relevance of the simulator modules. Six modules were selected for the test and a pilot study was carried out. Subsequently, medical students, residents, and experienced gynaecologists were enrolled for testing. Outcomes were based on generated simulator metrics. Validity evidence was explored for all five sources of evidence (content, response process, internal structure, relations to other variables, consequences of testing).
RESULTS: Inter-case reliability was high for four out of five metrics (Cronbach’s alpha ≥ 0.80). Significant differences were identified when comparing the three groups’ performances (p values < 0.05). Participants’ clinical experience was significantly correlated to their simulator test score (Pearson’s r = 0.49, p < 0.001). A single medical student managed to achieve the established pass/fail score (6.7% false positive) and three experienced gynaecologists failed the test (27.3% false negative).
CONCLUSIONS: We developed a virtual-reality simulation-based test in hysteroscopy with supporting validity evidence. The test is intended to ensure competency in a mastery learning program where trainees practise on the simulator until they are able to pass before they proceed to supervised training on patients.
Preliminary Experience with Virtual Reality Simulation vs. Animal Model for Hysteroscopic Myomectomy Training
L.R. Glazerman, S.R. Hart, M. Bajka, D. Fink, R.R. Bassaly
Glazerman LR, 1 Hart SR, 1 Bajka M, 2 Fink D, 2 Bassaly RR. 1 1Center for the Advancement of Minimally Invasive Pelvic Surgery, University of South Florida College of Medicine, Tampa, FL; 2Obstetrics and Gynecology, University Hospital of Zurich, Zurich, Switzerland
The Journal of Minimally Invasive Gynecology November 2009 (Vol. 16, Issue 6, Supplement, Page S56)
STUDY OBJECTIVE: To obtain trainees’ initial impressions of a new virtual reality hysteroscopic trainer compared to a traditional animal model for training in hysteroscopic myomectomy.
DESIGN: Attendees at a hands-on postgraduate course on operative hysteroscopy rotated through several training stations. Included in the rotation were a station using a pig bladder model for hysteroscopic myomectomy as well as a station using the HystSim (VirtaMed, Zurich, Switzerland) hysteroscopic simulator. Each participant rated both stations on realism as well as training capacity.
No significant difference was observed between the two models for realism
SETTING: Sixteen participants in a hands-on postgraduate course on operative hysteroscopy.
INTERVENTION: Immediately after a training session using either the pig bladder model or the HystSim simulator, participants were asked to rate each model on realism and training capacity using a 5-point Likert scale.
MEASUREMENTS AND MAIN RESULTS: The virtual reality trainer scored significantly higher than the pig bladder model for ‘‘variety of training cases’’ (p50.039) and ‘‘performance assessment’’ (p50.010). There was a statistically insignificant trend in favor of the virtual reality model for training capacity.
No correlation was found between age, experience, or years of practice.
CONCLUSION: The HystSim Virtual Reality hysteroscopic trainer was felt to be at least equal to the ‘‘gold standard’’ pig bladder model for training in hysteroscopic myomectomy with the additional advantages of reproducibility and measurement of results. Further studies comparing modalities and relating results to operating room performance are warranted.
Hysteroscopic placement of tubal sterilization implants: virtual reality simulator training
Pierre Panel, Michael Bajka, Arnaud Le Tohic, Alaa El Ghoneimi, Carmen Chis and Stéphane Cotin
Service de Gynécologie-Obstétrique, Centre Hospitalier de Versailles, Hôpital André Mignot, 177 avenue de Versailles, 78157, Le Chesnay, France
Surgical Endoscopy 2012, DOI: 10.1007/s00464-011-2139-6
STUDY OBJECTIVE: To assess face and construct validity of a new virtual reality (VR) training simulator for hysteroscopic placement of tubal sterilization implants.
DESIGN: Nonrandomized, controlled trial comparing responses and performance of novices and experts on the simulator. Classification: Canadian task force II-1.
SETTING: Forty-six gynecologists were personally invited or recruited at the 33rd Conference of the French National College of Gynecologists and Obstetricians (CNGOF) from December 9 to 12, 2009, grouped as 20 experts and 26 novices. They all performed the defined sequence of virtual procedures on the simulator (case 1 for familiarization, case 4 for study assessment) and finally completed the study questionnaire.
MEASUREMENTS AND MAIN RESULTS: Responses to realism, educational potential, and general opinion were excellent, proving face validity. Significant differences between novices and experts were assessed for 7 of the 15 metrics analyzed, proving construct validity.
CONCLUSIONS: We established face and construct validity for EssureSim™, an educational VR simulator for hysteroscopic tubal sterilization implant placement. The next steps are to investigate convergent and predictive validity to affirm the real capacity of transferring the skills learned on the training simulator to the patient in the operating room.
The following abstract was presented at the Annual Congress of the European Society for Gynaecological Endoscopy (ESGE), September 21-24, 2011 London, UK
Face & Construct Validity of a Virtual Reality Simulator for Hysteroscopy Essure Sterilization
J.A. Janse1, S. Veersema1, F. Broekmans, H.W.R.2 Schreuder2
1 St. Antonius Hospital Nieuwegein, Netherlands
2University Medical Centre Utrech, Netherlands
INTRODUCTION: The face and construct validity of a new virtual reality simulator for the hysteroscopy Essure sterilization method is established by determining the extent of realism of the simulation to the actual task and by analysis of the performance of participants with varying hysteroscopy experience.
OBJECTIVE: The aim of this study is to determine face and construct validity of the EssureSim™, a new virtual reality simulator for the Essure sterilization method.
METHODS: Residents and gynecologists (N=63) were recruited for voluntary participation. Three groups were made based on hysteroscopy experience: novices (N=10), intermediates (N=43) and experts (N=10). Participants completed three exercises. The second and third exercises were used for analysis. Parameters ‘time’, ‘path length’, ‘patient comfort’, ‘successful placement’, ‘trauma’ and ‘distension medium’ were compared between groups to determine construct validity. Afterwards, participants filled out a questionnaire addressing hysteroscopy experience and the simulator. By analysis of the questionnaire face validity was determined.
RESULTS: The experts performed both exercises significantly faster than the novice group (p<.05). The novices had a longer path length in the first exercise in comparison to the intermediate and expert group (p<.05). Analysis of the other parameters did not show any significant results between groups. In the questionnaire realism and training capacity of the EssureSim™ were both scored with a median of 4 points on a 5-point Likert scale. Of all participants, 96.8% indicated the EssureSim™ as a useful preparation for real-time Essure placement.
CONCLUSION: Face and construct validity was established of the EssureSim™ and this simulator is considered to be a useful training method for different levels of expertise. Virtual reality simulation could offer a significant contribution to the training of hysteroscopy skills.
Establishing Construct Validity of a Virtual-Reality Training Simulator for Hysteroscopy Via a Multimetric Scoring System
Michael Bajka Æ Stefan Tuchschmid Æ Daniel Fink Æ Ga´bor Sze´kely Æ Matthias Harders
Clinic of Gynaecology, Dept. OB/GYN, University Hospital Zurich, 8091, Zurich, Switzerland
Surg Endosc. 2009 June 24.
BACKGROUND: The aims of this study are to determine construct validity for the HystSim virtual-reality (VR) training simulator for hysteroscopy via a new multimetric scoring system (MMSS) and to explore learning curves for both novices and experienced surgeons.
METHODS: Fifteen relevant metrics had been identified for diagnostic hysteroscopy by means of hierarchical task decomposition. They were grouped into four modules
(visualization, ergonomics, safety, and fluid handling) and individually weighted, building the MMSS for this study. In a first step, 24 novice medical students and 12 experienced gynecologists went through a self-paced teaching tutorial, in which all participants received clearly stated goals and instructions on how to carry out hysteroscopic procedures properly for this study. All subjects performed five repeated trials on two different exercises on HystSim (exploration and diagnosis exercises). After each trial the results were presented to the participants in the form of an automated objective feedback report (AOFR). Construct validity for the MMSS and learning curves were investigated by comparing the performance between novices and experienced surgeons and in between the repeated trials. To study the effect of repeated practice, 23 of the novices returned 2 weeks later for a second training session.
RESULTS: Comparing novices with the experienced group,the ergonomics and fluid handling modules resulted in construct validity, while the visualization module did not,
and for the safety module the experienced group even scored significantly lower than novices in both exercises. The overall score showed only construct validity when the
safety module was excluded. Concerning learning curves, all subjects improved significantly during the training on HystSim, with clear indication that the second training session was beneficial for novice surgeons.
CONCLUSIONS: Construct validity for HystSim has been established for different modules of VR metrics on a new MMSS developed for diagnostic hysteroscopy. Careful
refinement and further testing of metrics and scores is required before using them as assessment tools for operative skills.
Evaluation of a New Virtual-Reality Training Simulator for Hysteroscopy
Bajka M, Tuchschmid S, Streich M, Fink D, Székely G, Harders M.
Clinic of Gynaecology, Dept. OB/GYN, University Hospital Zurich, 8091, Zurich, Switzerland
Surg Endosc. 2008 Apr 24.
BACKGROUND: To determine realism and training capacity of HystSim, a new virtual-reality simulator for the training of hysteroscopic interventions.
METHODS: Sixty-two gynaecological surgeons with various levels of expertise were interviewed at the 13(th) Practical Course in Gynaecologic Endoscopy in Davos, Switzerland. All participants received a 20-min hands-on training on the simulator and filled out a four-page questionnaire. Twenty-three questions with respect to the realism of the simulation and the training capacity were answered on a seven-point Likert scale along with 11 agree-disagree statements concerning the HystSim training in general.
RESULTS: Twenty-six participants had performed more than 50 hysteroscopies (“experts”) and 36 equal to or fewer than 50 (“novices”). Four of 60 (6.6%) responding participants judged the overall impression as “7 – absolutely realistic”, 40 (66.6%) as “6 – realistic”, and 16 (26.6%) as “5 – somewhat realistic”. Novices (6.48; 95% confidence interval [CI] 6.28-6.7) rated the overall training capacity significantly higher than experts (6.08; 95% CI 5.85-6.3), however, high-grade acceptance was found in both groups. In response to the statements, 95.2% believe that HystSim allows procedural training of diagnostic and therapeutic hysteroscopy, and 85.5% suggest that HystSim training should be offered to all novices before performing surgery on real patients.
CONCLUSION: Face validity has been established for a new hysteroscopic surgery simulator. Potential trainees and trainers assess it to be a realistic and useful tool for the training of hysteroscopy. Further systematic validation studies are needed to clarify how this system can be optimally integrated into the gynaecological curriculum.