Effect of a Localized ICU-Specific AI Teaching Agent on Institutional Workflow Mastery and Clinical Competency in Rotating ICU Residents: A Single-Center, Parallel-Group, Randomized Controlled Superiority Trial
试验速览
- 阶段
- 不适用
- 状态
- 招募中
- 入组人数
- 44
- 试验地点
- 1
- 主要终点
- Total task completion time for standardized ICU protocol task battery
研究概览
简要总结
This trial is an ongoing single-center, pragmatic, parallel-group randomized controlled superiority trial currently in participant recruiting phase, conducted within intensive care unit teaching wards at Peking Union Medical College Hospital, Beijing, China. The scheduled trial implementation period spans March 2026 to June 2026, aiming to evaluate whether an institution-specific, protocol-bound retrieval-augmented AI educational agent (named ICU-Tutor) can reduce residents' extraneous cognitive load and improve standardized ICU protocol task performance compared with free access to unrestricted commercial general-purpose large language model AI tools during early ICU clinical rotation.
The trial plans to screen a total of 44 first-time ICU rotating resident candidates, with pre-defined exclusion standards to eliminate unqualified individuals; approximately 44 eligible residents will undergo 1:1 stratified randomization and be split into two research arms: 22 participants assigned to the ICU-Tutor intervention group and 22 assigned to the unrestricted general AI control group.
All enrolled subjects will complete standardized 14-day follow-up assessments as pre-specified in the trial protocol. Both study cohorts receive unified 15-minute standardized training covering standardized safe AI clinical application rules prior to formal intervention initiation. ICU-Tutor is strictly built on a curated knowledge base including 247 ICU institutional protocols validated by senior attending intensivists, with all AI outputs traceable back to original local protocol documents and constrained within verified institutional guidance content only. The control arm allows participants to select and utilize any mainstream general large-model AI tools per personal preference without content or access limitations, consistent with real-world daily resident clinical practice.
Two co-primary endpoints are uniformly scheduled to be measured on the 7th day after randomization, including total completion duration of standardized ICU protocol task battery and Paas 9-point validated cognitive load scale score reflecting participants' subjective mental workload during task execution. Three confirmatory secondary endpoints are pre-defined for centralized assessment: composite task performance score on Day7, written institutional protocol knowledge retention score tested on Day14, and 0-100-point visual analog scale (VAS) evaluating resident satisfaction toward allocated AI support on Day7. Individual sub-station scores of three split practical ICU skill modules are set as exploratory secondary endpoints for post-hoc descriptive analysis only.
The statistical analysis framework is pre-specified to follow intention-to-treat principle entirely. Analysis of covariance (ANCOVA) is selected as core analytical method for all continuous outcomes, with Day3 baseline assessment result and participants' academic training background set as pre-planned covariates. Bonferroni multiple-testing correction is applied for dual co-primary endpoints, while Benjamini-Hochberg false discovery rate (FDR) correction is pre-specified to control type I error across three confirmatory secondary outcomes. Effect sizes will be quantified via Cohen's d after raw data collection and database lock.
The trial has obtained formal ethical approval from the Institutional Review Board of Peking Union Medical College Hospital (Approval ID: I-26ZM0024). Every enrolled resident provides written informed consent before random assignment.
详细描述
- Research Background Early-stage intensive care unit (ICU) clinical rotation imposes substantial cognitive challenges for newly incoming resident physicians. Novice residents must adapt to unfamiliar ward workflows, manage critically ill patients with unstable vital status under severe time constraints, and master a large volume of hospital-specific institutional clinical protocols covering routine and emergency ICU management. Rooted in Cognitive Load Theory, unnecessary extraneous cognitive workload originating from repeated information searching and cross-verification severely occupies limited working memory capacity and hinders clinical learning efficiency during high-intensity residency training.
General-purpose large language model (LLM) AI tools have gained widespread popularity as supplementary learning resources among medical trainees worldwide. Nevertheless, conventional off-the-shelf LLMs lack embedded access to hospital-specific, site-validated ICU clinical protocols, which leads to generalized, decontextualized recommendations inconsistent with local institutional practice requirements. Consequently, residents are forced to spend extra working memory to validate AI-generated suggestions against internal hospital guidelines, generating avoidable cognitive burden that impedes on-the-job protocol learning and bedside task execution.
Retrieval-augmented generation (RAG) framework enables customized institutional AI agents bounded exclusively within locally approved ICU protocols, delivering source-cited, site-compliant clinical guidance without requiring post-hoc manual verification by trainees. Existing medical education AI research predominantly evaluates model performance on generalized medical knowledge examinations rather than real-world on-site protocol application within authentic ICU working environments. This prospective randomized controlled trial is designed to fill this research gap by comparing the educational benefits of a hospital-customized protocol-locked AI agent (ICU-Tutor) versus unrestricted free access to commercial general LLMs among first-time ICU rotating residents during a standardized 14-day observation window. The core research hypothesis specifies that ICU-Tutor will lower resident cognitive load and improve protocol-based task performance relative to open-access general AI. 2. Predefined Study Objectives Primary Objectives To prospectively evaluate whether ICU-Tutor reduces standardized task completion time and self-reported Paas-scale cognitive load on study Day 7 compared with unrestricted general-purpose AI access in first-time ICU rotating residents. These two metrics serve as co-primary trial endpoints for formal statistical comparison.
Confirmatory Secondary Objectives
Three pre-specified secondary endpoints will undergo formal statistical testing with pre-defined multiple-testing correction rules:
研究设计
- 研究类型
- Interventional
- 分配方式
- Randomized
- 干预模型
- Parallel
- 主要目的
- Other
- 盲法
- Single (Outcomes Assessor)
入排标准
- 年龄范围
- 18 Years 至 40 Years（Adult）
- 性别
- All
- 接受健康志愿者
- 是
入选标准
- •First-time rotating residents with no prior formal ICU clinical rotation experience
- •Scheduled to complete a minimum of 4 consecutive weeks of ICU training
- •Able to complete all scheduled assessments at Day 3, Day 7, and Day 14 post-randomization
- •Possess basic digital literacy to operate assigned AI tools on standard clinical devices
- •Voluntarily provide written informed consent to participate in the trial
排除标准
- •Cumulative prior formal ICU clinical experience exceeding 7 calendar days
- •Regular daily clinical use of AI tools for medical decision-making in the 3 months preceding enrollment
- •Physical or cognitive impairment that prevents completion of trial assessments
- •Inability to provide written informed consent
结局指标
主要结局
Total task completion time for standardized ICU protocol task battery
时间窗: Day 7 post-randomization
Cumulative time (in minutes) required to complete a 3-station standardized practical assessment covering core ICU protocol applications
Paas 9-point cognitive load scale score
时间窗: Day 7 post-randomization
Validated single-item subjective rating of mental workload during task completion, ranging from 1 (extremely low mental effort) to 9 (extremely high mental effort)
次要结局
- Composite practical task performance score(Day 7 post-randomization)
- AI tool satisfaction visual analogue scale (VAS) score(Day 7 post-randomization)