Effect of Artificial Intelligence-Augmented Human Instruction on Surgical Simulation Performance: A Randomized Controlled Trial
试验速览
- 阶段
- 不适用
- 状态
- 已完成
- 入组人数
- 88
- 试验地点
- 2
- 主要终点
- Intelligent Continuous Expertise Monitoring System (ICEMS) expertise score - Technical skill acquisition across practice tasks on NeuroVR simulator
研究概览
简要总结
At the Neurosurgical Simulation and Artificial Intelligence Learning Centre, we seek to provide surgical trainees with innovative technologies that allow them to improve their surgical technical skills in risk-free environments, potentially improving patient operative outcomes. The Intelligent Continuous Expertise Monitoring System (ICEMS), a deep learning application that assesses and trains neurosurgical technical skill and provides continuous intraoperative feedback, is one such technology that may improve surgical education.
In this randomized controlled trial, medical students from four Quebec universities will be blinded and randomized to one of three groups (one control and two experimental). Group 1 (control) will be provided with verbal AI tutor feedback based on the ICEMS error detection. Group 2 will be tutored by a human instructor who will receive ICEMS error data and deliver verbal instruction using the same words as the ICEMS. Group 3 will be tutored by a human instructor who will be provided with ICEMS data and will then deliver personalized feedback.
The aim of this study is to determine how the method of delivery of verbal surgical error instruction influences trainee technical skill acquisition and transfer. Evaluating trainee responses to AI instructor verbal feedback as compared to feedback from human instructors will allow for further development, testing, and optimization of the ICEMS and other AI tutoring systems.
详细描述
Background: Expert surgical technical skill is linked with improved patient outcomes; however, training novices to master these skills remains challenging. The Intelligent Continuous Expertise Monitoring System (ICEMS) is a deep learning application that was developed at the Neurosurgical Simulation and Artificial Intelligence Learning Centre to improve neurosurgical education. The ICEMS assesses and trains bimanual surgical performance by providing continuous feedback via verbal instructions in order to improve trainee performance and mitigate errors.
Rationale: A previous randomized controlled trial (RCT) performed at our centre demonstrated that intelligent tutoring is more effective than expert tutoring in a simulated neurosurgical procedure (NCT05168150). However, during this study, expert instructors were not provided with ICEMS error data. Conducting a new RCT in which expert instructors are provided with quantitative ICEMS error data will allow us to determine the most effective method for teaching surgical technical skills to trainees in virtual operative procedures.
This report follows the Consolidated Standards of Reporting Trials-Artificial Intelligence (CONSORT-AI) as well as the Machine Learning to Assess Surgical Expertise (MLASE) checklist.
Hypotheses:
- AI-augmented personalized expert instruction will be more effective at improving technical skill acquisition and skill transfer in trainees compared to AI tutor instruction.
- Instruction delivered by human instructors will result in lower levels of negative emotions and cognitive load compared with instruction delivered by the AI tutor.
研究设计
- 研究类型
- Interventional
- 分配方式
- Randomized
- 干预模型
- Parallel
- 主要目的
- Health Services Research
- 盲法
- Single (Participant)
盲法说明
Single (Participant)
Study participants are blinded to group assignments and study outcomes.
入排标准
- 年龄范围
- 18 Years 至 —（Adult, Older Adult）
- 性别
- All
- 接受健康志愿者
- 是
入选标准
- •Medical students who are actively enrolled in their preparatory, first, second year of medical school at any Quebec institution who do not fit the exclusion criteria.
排除标准
- •Prior use of the NeuroVR (CAE Healthcare) simulator.
研究组 & 干预措施
AI tutor instruction group
31 participants allocated. During their second, third, fourth, and fifth repetition of the practice subpial brain tumor resection scenario, participants will receive verbal ICEMS feedback when the system detects an error on their performance.
Expert instruction group
29 participants allocated. During their second, third, fourth, and fifth repetition of the practice subpial brain tumor resection scenario, participants will receive verbal feedback from an expert instructor. The expert instructor will deliver this feedback using the same words as the ICEMS.
干预措施: Expert instruction using AI tutor script (Behavioral)
Personalized expert instruction group
28 participants allocated. During their second, third, fourth, and fifth repetition of the practice subpial brain tumor resection scenario, participants will receive verbal feedback from an expert instructor. The expert instructor will use their expertise to deliver personalized feedback to the participant.
干预措施: AI-augmented personalized expert instruction (Behavioral)
结局指标
主要结局
Intelligent Continuous Expertise Monitoring System (ICEMS) expertise score - Technical skill acquisition across practice tasks on NeuroVR simulator
时间窗: 1 day of study
The ICEMS will continuously evaluate the trainee's performance during each practice task and calculate average expertise scores on a scale of -1.00 (novice) to 1.00 (expert). This will allow us to assess learner technical skill acquisition from the first through sixth repetitions of the practice task.
Intelligent Continuous Expertise Monitoring System (ICEMS) expertise score - Technical skill transfer during complex realistic task on NeuroVR simulator
时间窗: 1 day of study
The ICEMS will continuously evaluate the trainee's performance during the realistic task and calculate an average expertise score on a scale of -1.00 (novice) to 1.00 (expert). This will allow us to assess learner technical skill transfer from the practice tasks to a more complex realistic scenario.
次要结局
- Strength of emotions elicited(1 day of study)
- Levels of cognitive load(1 day of study)
研究者
Rolando Del Maestro
Director of the Neurosurgical Simulation and Artificial Intelligence Learning Centre
McGill University