TURP Mentor Clinical Validation

Development and validation of a simulator-based test in transurethral resection of bladder tumors establishing pass/fail standards

Sarah Bube, Rikke Hansen, Claus Dahl, Lars Konge, and Nessn Azawi


INTRODUCTION AND OBJECTIVES: To develop a simulator-based test with ability to discriminate between competency levels and establish a pass/fail score for proficiency-based simulator training in TURB.

METHODS: Forty-nine doctors were recruited from Departments of Urology in the Oresund Region, Denmark in the period of Apr to Sept 2018. Prior to inclusion, the TURB Mentor TM virtual reality (VR) simulator, Simbionix, was assessed at an expert meeting selecting clinically relevant cases and metrics. Training sessions were done on identical simulators at two simulation training centers at Department of Urology, Roskilde Zealand University Hospital and Copenhagen Academy for Medical Education and Simulation. All participants performed three TURB procedures on the VR simulator. Simulator metrics were analyzed with analysis of variance (ANOVA) and metrics with ability to discriminate between groups were combined in a total simulator score (TSS). Finally, a pass/fail score was identified using the contrasting groups’ method.

RESULTS: Elleven simulator metrics were found eligible of which four had significant discrimination ability between the three competency levels: resected pathology (%) (p=0.008), cutting in bladder wall (n) (p=0.004), time (s) (p=0.034), and inspection of bladder wall (%) (p=0.002). The internal structure of the TSS [(resected pathology*inspection of bladder wall)/time] was acceptable with intraclass correlation, Cronbach’s α: 0.85. The mean TSS was significantly lower in the novice group than in the intermediates (p=0.011) and experienced group (p<0.001) (Table 1). A pass/fail score of 21 was identified (Fig 1). CONCLUSIONS: Validity evidence for a VR simulator-based test has been established including a pass/fail score identifying TURB competence. The test can be used for accreditation in a proficiency-based TURB simulator training program prior to supervised procedures.

Visual control strategies of surgeons: a novel method of establishing the construct validity of a transurethral resection of the prostate surgical simulator.

Bright E1, Vine SJ2, Dutton T3, Wilson MR2, McGrath JS3.

1Exeter Surgical Health Services Research Unit, Royal Devon & Exeter NHS Foundation Trust, Devon, United Kingdom. Electronic address: ebright9@gmail.com.

2College of Life and Environmental Sciences, University of Exeter, St Luke’s Campus, Devon, United Kingdom.

3Exeter Surgical Health Services Research Unit, Royal Devon & Exeter NHS Foundation Trust, Devon, United Kingdom.

J Surg Educ. 2014 May-Jun;71(3):434-9.

OBJECTIVE: To examine novice and expert differences in visual control strategies while performing a virtual reality transurethral resection of the prostate (TURP) task and to determine if these differences could provide a novel method for assessing construct validity of the simulator.

SUBJECTS AND METHODS: A total of 11 novices (no TURP experience) and 7 experts (>200 TURPs) completed a virtual reality prostate resection task on the TURPsim (Simbionix USA Corp, Cleveland, OH) while wearing an eye tracker (ASL, Bedford, MA). Performance parameters and the surgeon’s visual control strategy were measured and compared between the 2 groups.

RESULTS: Experts resected a greater percentage of prostate than novices (p < 0.01) and had less active diathermy time without tissue contact (p < 0.01). Experts adopted a target-locking visual strategy, employing fewer visual fixations (p < 0.05) with longer mean fixation duration (p < 0.005). With multiple learning trials, novices' performance improved and the adoption of a more expertlike gaze strategy was observed. CONCLUSION: Significant differences between experts and novices in both performance and visual control strategy were observed. The study of visual control strategies may be a useful adjunct, alongside measurements of motor performance, providing a novel method of assessing the construct validity of surgical simulators.

Evaluation of the educational value of a virtual reality TURP simulator according to a curriculum-based approach.

Tjiam IM1, Berkers CH, Schout BM, Brinkman WM, Witjes JA, Scherpbier AJ, Hendrikx AJ, Koldewijn EL.

Catharina Hospital Eindhoven, Eindhoven ,Medical Centre Alkmaar (B.M.S.); EMGO Institute (B.M.S.), VU Medical Center, Alkmaar and Amsterdam; Radboud University Nijmegen Medical Centre, Nijmegen (J.A.W.); and Maastricht University, Maastricht (A.J.S.), the Netherlands.

Simul Healthc. 2014 Oct;9(5):288-94

PURPOSE: This study aimed to evaluate the place of the TURPsim (Simbionix Beit Golan, Israel) within a urologic residency training curriculum, including training needs analysis (TNA) and investigating its validity.

MATERIALS AND METHODS: Training needs analysis was conducted by an expert panel to identify procedural steps and pitfalls. Performance metrics of the simulator were compared with the TNA results. Participants were distributed according to their level of experience (completed transurethral resection of the prostate [TURP] procedures) as follows: novices (0), intermediates (1-50), and experts (>50). They followed standardized instructions and then performed 2 completeTURP procedures on the TURPsim.

RESULTS: Ten of 22 procedural steps (TNA) and 4 of 11 pitfalls were covered by the TURPsim. A total of 66 participants, 22 in each group, were included. Median general judgment (face and content) about the TURPsim was rated 7.3 (median, 7; range, 3-9). Ninety-three percent of all participants qualified the TURPsim as a useful training model. Intermediates and experts had a significant faster resection time and less blood loss compared with novices (construct) (P = 0.001). Novices needed to re-resect previous lobes, and they also resected the prostate in the incorrect order more frequently compared with intermediates and experts.

CONCLUSIONS: Training needs analysis is of paramount importance in the evaluation process of a training program. This curriculum-based approach including validity of a simulator seems valuable and may narrow the gap between skills laboratory and clinical practice. This study showed face, content, and construct validity of the TURPsim, and this simulator finds its place in the current urologic curriculum to train basic and procedural TURP skills.

The following abstract was presented at the annual meeting of the American Urological Association, May 19-23, 2012, Atlanta, Georgia

Examining the visual control strategies of experts and novices to establish the validity of a novel TURP simulator.

Thomas Dutton; Samuel Vine; John McGrath; Elizabeth Bright; Mark Wilson

INTRODUCTION AND OBJECTIVES: With increasing resource constraints and patient concern, there has been a recent interest in the use of simulators to help surgeons acquire technical skills. Surgical simulators facilitate repetitive practice and performance feedback in a risk-free environment. Previous validation of simulators has focussed on motor proficiency measures. However, studies have shown that expertise in laparoscopic skills is underpinned by an efficient visual control strategy (fixations that are long in duration and low in frequency). Using infrared eye-tracking devices, experts are seen to ‘lock’ their gaze on a target earlier and for longer, improving the accuracy of the subsequent motor skill. The aim of this study was to examine visual control measures and their utility as a novel method to assess the construct validity of a transurethral resection of prostate (TURP) virtual reality simulator (VR; Simbionix, USA Corp).

METHODS: 11 novices (no TURP experience) and 7 experts (>200 TURPS) completed the median lobe resection task on the simulator. All participants wore a gaze registration system that assessed their visual control strategy (recorded as the number of fixations per second and mean duration of fixations). The VR simulator also provided objective measures of performance (% prostate resected per minute) and safety/error (% prostatic capsule perforation and time diathermy active without tissue contact).

RESULTS: When compared to novices, experts employed a visual control strategy of fewer fixations per second (mean 1.49 vs 1.12, p < 0.05), but a longer mean fixation duration (0.43 vs 0.76, p < .005). Differences were also found in performance related measures; experts resected 59.05% of the prostate per minute whereas novices resected only 30.86% (p < .001) and experts were more precise with the diathermy than novices (0.32 vs 2.35, p < .005). However, both groups had an equal rate of prostatic capsular perforation (13.71 vs 16.91, p = 0.31).

CONCLUSIONS: Significant differences have been shown in the performance of experts versus novices (proficiency and safety parameters) whilst performing a VR TURP training task. Furthermore, experts and novices displayed the expected differences in visual control strategies. This suggests that the TURP simulator is a valid tool in the replication of this common urological procedure. The study of visual control strategies may be a useful adjunct, alongside assessment of motor performance, in assessing surgeons undergoing simulation training.

This work was supported by a bilateral research_x000D_ grant from the Economic and Social Research Council, UK and the Research Grants Council, Hong Kong (RES-000-22-3016).

The following abstract was presented at the annual meeting of the American Urological Association, May 19-23, 2012, Atlanta, Georgia, USA

Learning effects using a TURP simulator: Assessing changes in visual control and performance.

Samuel Vine; Thomas Dutton; Mark Wilson; Elizabeth Bright; John McGrath

INTRODUCTION AND OBJECTIVES: Surgical simulators afford trainees the chance to practise skills in a safe environment and without the need for supervision. Although they have been proposed to shorten the learning curve for complex surgical skills, there is concern that they do not prepare trainees for the demanding conditions of the operating room. Research in skill learning (including surgical skills) has shown that experts and novices can be distinguished by differences in their visual control strategies, with experts using fewer fixations of a longer duration. The aim of this study was to assess the learning benefits of a TURP simulator by examining, not only changes in novice performance, but also changes in their visual control.

METHODS: 11 novices (no TURP experience) completed a basic training task on a virtual reality TURP simulator (Simbionix, USA Corp) whilst wearing an eye tracker to measure their visual control (number and duration of fixations). Performance measures (% prostate & % capsule resected per second) were provided by the simulator. All participants performed a baseline trial, 5 learning trials, a retention trial (to asses learning) and a tone counting trial (to mimic the multi-tasking demands of the operating theatre). All data were subjected to one-way repeated measures ANOVA and significant effects were followed up with pair-wise comparison tests.

RESULTS: Results revealed a significant improvement in performance from baseline to retention, with increases in the % of prostate resected (p < .001) and reductions in the % of capsule resected (p < .005). There was also a significant improvement in visual control from baseline to retention, with reductions in the number (p < .005) and duration of fixations (p < .05). However, there was a significant breakdown in both performance and visual control under multi tasking conditions (p’s < .005).

CONCLUSIONS: After only five repetitions of a simple training task, novices became more proficient (% prostate resected) and less error full (% capsule resected), supporting the use of simulation for surgical skills training. The novices also developed more ‘expert like’ visual control, which has been shown to underpin technical skill performance. However, these changes were not robust under multi-tasking conditions, highlighting the limitations of simulated training. Future research should examine training interventions designed to ensure that visual control and motor skill performance remain robust in stressful conditions.

This work was supported by a bilateral research_x000D_ grant from the Economic and Social Research Council, UK and the Research Grants Council, Hong Kong (RES-000-22-3016).

Face validity, construct validity and training benefits of a virtual reality TURP simulator.

Bright E, Vine S, Wilson MR, Masters RS, McGrath JS.

Exeter Surgical Health Services Research Unit, Royal Devon & Exeter NHS Foundation Trust, Barrack Road, Exeter, Devon EX2 5DW, UK.

Int J Surg. 2012;10(3):163-6. Epub 2012 Feb 23.

OBJECTIVE: To assess face validity, construct validity and the training benefits of a virtual reality TURP simulator.

METHOD: 11 novices (no TURP experience) and 7 experts (>200 TURP’s) completed a virtual reality median lobe prostate resection task on the TURPsim™ (Simbionix USA Corp., Cleveland, OH). Performance indicators (percentage of prostate resected (PR), percentage of capsular resection (CR) and time diathermy loop active without tissue contact (TAWC) were recorded via the TURPsim™ and compared between novices and experts to assess construct validity. Verbal comments provided by experts following task completion were used to assess face validity. Repeated attempts of the task by the novices were analysed to assess the training benefits of the TURPsim™.

RESULTS: Experts resected a significantly greater percentage of prostate per minute (p < 0.01) and had significantly less active diathermy time without tissue contact (p < 0.01) than novices. After practice, novices were able to perform the simulation more effectively, with significant improvement in all measured parameters. Improvement in performance was noted in novices following repetitive training, as evidenced by improved TAWC scores that were not significantly different from the expert group (p = 0.18).

CONCLUSIONS: This study has established face and construct validity for the TURPsim™. The potential benefit in using this tool to train novices has also been demonstrated.