Effect of Intelligent Tutor Induced Pausing on Learning Simulated Surgical Skills Among Medical Students: A Randomized Controlled Trial
试验速览
- 阶段
- 不适用
- 状态
- 招募中
- 入组人数
- 129
- 试验地点
- 2
- 主要终点
- Change in performance
研究概览
简要总结
Traditional training of surgical technical skills relies on mentorship from experienced surgeons, who continuously evaluate and change trainee performance to prevent errors and potential patient harm by providing verbal instructions. These educators may also pause the procedure, explaining the risks associated with the trainee's actions, and may personally demonstrate proper techniques to the students. Studies examining pausing while providing medical care outline that these approaches allow for learning.
An artificial intelligent (AI) tutoring system, the Intelligent Continuous Expertise Monitoring System (ICEMS), improves learning in a surgical simulated operation by providing trainees with verbal instructions upon error identification. However, the effect of including a pause during this AI teaching has not been studied. Therefore, the ICEMS post-error identification methodology has been altered to include a pause with the intelligent tutor voice instruction.
The aim of this study is to determine the effect of pausing on surgical skill acquisition and transfer among pre-medical and medical students. This will be done by comparing their performance in repeated simulated tumour resection tasks.
详细描述
Background: Surgical skill assessment is shifting from a quantitative, time-based approach towards a qualitative evaluation of a trainee's competency. During surgical procedures, instructors continuously monitor trainee performance and utilize various teaching methods focused on enhancing acquisition of surgical skills. One such method includes pausing the operation, either to outline the risks associated with the trainee's performance or to personally demonstrate the best practice technique(s). Pausing in such situations has been shown to allow learners to re-assess best practice, interrupt negative momentum, and allow for learning. Specifically, pausing after an error can prevent introduction of new information that may affect one's ability to reflect on their error and reduce stress before continuing.
Rationale: The ICEMS, an AI tutoring system, was developed by our group using a Long-Short Term Memory deep learning algorithm to assess surgical performance and provide guidance. This was then integrated with the NeuroVR simulation platform. Using this AI system, the provision of verbal feedback on error identification demonstrated the potential of intelligent tutoring to improve learning in two previous randomized control trials (RCTs). However, these RCTs did not incorporate pausing methodology post-error identification. To further emulate the mentorship of an experienced surgeon in a clinical setting, the ICEMS platform has been modified to both initiate pausing when learner error is identified and provide a video demonstrating expert performance.
Research aims: To compare the effect of incorporating a pause after intelligent tutor instruction to intelligent tutor instruction alone on medical and pre-medical students' surgical skill acquisition and skill transfer.
Hypotheses:
- The pause with video group will significantly improve the composite score between the first and sixth repetition of the practice scenario.
- The pause with video group will have a composite score statistically higher than the control group in the sixth repetition of the practice scenario.
- The pause with video group will have a composite score statistically non-inferior to the pause without video group in the sixth repetition of the practice scenario.
- The pause with video group will have a global OSATS score statistically higher than the control group in the realistic scenario.
- The pause with video group will have a global OSATS score statistically non-inferior to the pause without video group in the realistic scenario.
- The pause with video group will not elect a difference in emotional stress or cognitive load compared to the control group.
研究设计
- 研究类型
- Interventional
- 分配方式
- Randomized
- 干预模型
- Parallel
- 主要目的
- Health Services Research
- 盲法
- Double (Participant, Outcomes Assessor)
盲法说明
Double (Participant and Expert Rater)
Participants are unaware of the intervention being used in the study - the pause. They are informed that they will be acquiring and honing technical skills relevant to neurosurgery, with the added feature of receiving feedback from an intelligent system during subpial tumour resection simulations.
Experts, when providing OSATS ratings, do not know to which group the video performance they are rating belongs.
入排标准
- 年龄范围
- 18 Years 至 —（Adult, Older Adult）
- 性别
- All
- 接受健康志愿者
- 是
入选标准
- •First and second year medical students who are actively enrolled in any Quebec institution who do not meet the exclusion criteria.
- •Students actively enrolled in medical school in a preparatory year in any Quebec institution who do not meet the exclusion criteria.
排除标准
- •Participation in previous trials involving the NeuroVR (CAE Healthcare) simulator
结局指标
主要结局
Change in performance
时间窗: 1 day of study
Evaluated by comparing the average composite-score, calculated by the ICEMS, from each practice scenario. Scores range from expert/skilled level (a score of 1.00) to novice/less-skilled level (a score of -1.00).
Objective Structured Assessment of Technical Skills (OSATS) global rating scale
时间窗: 1 day of study
Performance score of the participants in the complex realistic scenario, assessed by two blinded experts using the Objective Structured Assessment of Technical Skills (OSATS) global rating scale on a 7-point Likert scale (1= novice to 7 = expert). Efficacy in learning with pausing methodology and an expert-level demonstration video will be compared to pausing methodology alone and to no pausing methodology.
Transfer of learning
时间窗: 1 day of study
Evaluated by comparing the average composite-score, calculated by the ICEMS, from each practice scenario. Scores range from expert/skilled level (a score of 1.00) to novice/less-skilled level (a score of -1.00).
次要结局
- Differences in strength of emotions elicited(1 day of study)
- Difference in Cognitive Load(1 day of study)
研究者
Rolando Del Maestro
Director, Neurosurgical Simulation and Artificial Intelligence Learning Centre
McGill University