Developing and Gathering Validity Evidence for a Simulation-Based Test of Competencies in Lung Ultrasound

Pietersen PI1,2,3, Konge L4, Graumann O5,6, Nielsen BU7, Laursen CB8,5

Regional Center for Technical Simulation, Region of Southern Denmark, Odense, Department of Respiratory Medicine, Odense University Hospital, Odense, Institute for Clinical Research, University of Southern Denmark, Odense, Copenhagen Academy for Medical Education and Simulation, Central Region of Denmark, Copenhagen, Denmark. Institute for Clinical Research, University of Southern Denmark, Odense, Denmark. Department of Radiology, Odense University Hospital, Odense, Denmark. Department of Anaesthesiology and Intensive Care, Odense University Hospital, Odense, Denmark. Department of Respiratory Medicine, Odense University Hospital, Odense, Denmark.

BACKGROUND: Clinical lung ultrasound (LUS) is a fast bedside diagnostic tool which can assist clinicians in decisions regarding the treatment and monitoring of patients with respiratory symptoms. LUS training and education differ widely, and is often done in a clinical setting, with potential risks for patients if decisions are made based on the wrong interpretations. No clear guidelines or recommendations for objective and standardized assessment of LUS skills exist, and those that do are often based on a fixed time-frame or an arbitrary number of examinations performed; this does not ensure adequate competencies.

OBJECTIVES: The study aimed to develop and gather validity evidence for a practical, simulation-based test in LUS.

METHODS: Nine cases were developed in collaboration with 3D Systems Healthcare, Littleton, CO, USA, representing the most common diagnosis and sonographic findings in patients with respiratory symptoms. Thirty-six participants with different levels of competence in LUS, completed the test. The participants were divided into groups, i.e., novices, intermediates, and experienced, according to their experience with LUS, the number of examinations they had performed, and any research they had conducted. Their answers were used for item analyses.

RESULTS: The intraclass correlation coefficient, Cronbachs’ α, was 0.69 summarized, and there was a statistically significant difference (p < 0.001) between the novices and the trained participants (intermediates and experienced). A pass/fail score of 16 points was calculated according to the contrasting-groups method.

CONCLUSION: We developed a test for the assessment of clinical competencies in LUS. The test proved solid validity evidence, and a pass/fail standard without any false-negatives, and only 2 explained false-positives.

Training Pulmonary/Critical Care Fellows in Advanced Critical Care Transesophageal Echocardiography: A Simulator Based Training Project

Astha Chichra, MD; Seth Koenig, MD; Viera Lakticova, MD; Mangala Narasimhan, DO; Paul Mayo, MD

Chest. 2015;148(4_MeetingAbstracts):455A. doi:10.1378/chest.2280908

Sunday, October 25, 2015 at 04:30 PM – 05:30 PM

PURPOSE: Critical care transesophageal echocardiography (cTEE) is an important component of advanced critical care echocardiography. Experiential training in cTEE requires greater than 30 real case patients to achieve competence1. The effectiveness of high fidelity TEE simulator (simulator) training to achieve competence is unknown. We studied the effectiveness of using a simulator to train pulmonary/critical care medicine fellows (fellows) in cTEE image acquisition and interpretation.

METHODS: We trained 12 fellows with no experience in cTEE using a simulator. Using a standardized cTEE image acquisition format, trainees performed 14 complete studies, with image acquisition testing on the 15th study. We used a validated system for scoring the operative and interpretive skills of the trainees 2. Once competent on the simulator1, they performed cTEEs under direct attending supervision on patients. Each patient cTEE was graded using the same scoring system as for the simulator. Successful training outcome was defined as a score of >35/401.

RESULTS: Twelve fellows received training with the simulator, consisting of a 3 hour training period. All fellows achieved perfect performance in image acquisition on the simulator at the 15th cTEE study. Eight fellows have further performed at least 5 patient cTEEs. Six fellows achieved a score of >35/40 on their first patient cTEE. The other 2 fellows required an average of 3 studies to achieve a score >35/40.

CONCLUSIONS: Training fellows to perform cTEE using a simulator is effective and may reduce the number of real patient cTEEs needed for training purposes.

CLINICAL IMPLICATIONS: Simulator training can accelerate the training curve for cTEE in fellows. Further studies need to be performed in order to establish the minimum number of cTEEs to be performed to achieve competency. References: 1.Charron, C et al. Number of supervised studies required to reach competence in advanced critical care transesophageal echocardiography.Intensive care medicine (2013): 1-6. 2. 2.Charron, C et al. Validation of a skills assessment scoring system for transesophageal echocardiographic monitoring of hemodynamics. Intensive care medicine 33.10 (2007): 1712-1718.

DISCLOSURE: The following authors have nothing to disclose: Astha Chichra, Seth Koenig, Viera Lakticova, Mangala Narasimhan, Paul Mayo

No Product/Research Disclosure Information