Application and Effectiveness of a Large Language Model-Based Educational Agent in Medical Education: A Study on the Machine Learning and Data Mining Course
试验速览
- 阶段
- 不适用
- 状态
- 招募中
- 发起方
- 入组人数
- 56
- 试验地点
- 1
- 主要终点
- Composite Academic Performance Score
研究概览
简要总结
The goal of this interventional study is to evaluate the effectiveness of a Large Language Model (LLM)-based educational AI Agent in graduate students (Masters and PhD) specializing in medicine or nursing who are enrolled in the "Machine Learning and Data Mining" course. The main questions it aims to answer are:
Does the use of an educational AI Agent improve students' academic performance and practical skills in machine learning compared to traditional methods?
Does the AI intervention enhance students' learning confidence, satisfaction, and cognitive engagement?
Researchers will compare students currently using the AI Agent (experimental group) to a historical control group (students from the previous cohort who did not use the AI tool) to see if the AI-assisted learning model leads to significantly higher learning achievements and better educational experiences.
Participants will:
Utilize the Teaching Agent for real-time answers to theoretical questions, personalized study planning, and knowledge reinforcement.
Engage with the Research Agent to assist with literature reviews, research design optimization, and academic writing structure.
Use the Practice Innovation Agent for guidance on coding, algorithm debugging, and applying machine learning models to medical data analysis projects.
详细描述
Background : Artificial Intelligence (AI) and data mining are becoming essential skills in modern medical and nursing research. However, traditional teaching methods for the graduate-level course "Machine Learning and Data Mining" often struggle to meet the personalized learning needs of students with varying technical backgrounds (e.g., programming, mathematics). To address this, this study introduces a custom-developed AI Educational Agent based on Large Language Models (LLMs) to serve as an intelligent teaching assistant.
Objectives: The primary objective is to evaluate the effectiveness of the AI Agent in improving learning outcomes, practical coding skills, and academic self-efficacy among medical and nursing graduate students. The study also aims to assess the feasibility and student satisfaction of integrating AI agents into the medical curriculum.
Study Design: This is a non-randomized interventional study utilizing a historical control design.
Study Design: This is a non-randomized interventional study utilizing a historical control design.
Experimental Group (Intervention): Students in the 2025-2026 academic year who will receive access to the AI Agent system.
研究设计
- 研究类型
- Interventional
- 分配方式
- Na
- 干预模型
- Single Group
- 主要目的
- Other
- 盲法
- None
入排标准
- 性别
- All
- 接受健康志愿者
- 是
入选标准
- •Medical graduate students from universities in the Guangdong-Hong Kong-Macao Greater Bay Area;
- •Graduate students who have taken the "Machine Learning and Data Mining" course;
- •Have completed the required prerequisite courses: "Medical Statistics" and "Nursing Research";
- •Capable of operating the AI Educational Agent system normally and willing to undergo relevant teaching interventions and assessments during the study period.
排除标准
- •Unwilling to use the AI education agent system, or refusing to allow the research team to collect their relevant data;
- •Students who cannot commit to the full duration of the course or have known scheduling conflicts that would prevent regular attendance;
- •Students who have previously enrolled in or audited this course in prior academic years to avoid learning effect bias
研究组 & 干预措施
AI Agent Intervention Group
Graduate students enrolled in the "Machine Learning and Data Mining" course during the 2025-2026 academic year. Participants in this group will utilize the custom-developed KGRAG-based AI Educational Agent system throughout the semester. The system includes three modules: a Teaching Agent for concept explanation, a Research Agent for academic writing support, and a Practice Innovation Agent for code generation and debugging
干预措施: KGRAG-based AI Educational Agent System (Other)
结局指标
主要结局
Composite Academic Performance Score
时间窗: After the intervention (at the end of the course, approximately week 3)
Assessed through the final cumulative course grade (range: 0-100), which indicates the student's overall mastery of machine learning concepts and applications. The score is calculated based on three weighted components: In-class Assignments (20%): Evaluations of regular assignments submitted via the course platform. Research Progress Paper (40%): A written paper on a free-exploration topic assessing theoretical understanding and research design skills. Group Final Project Presentation (40%): Assessment of a practical project where students present solutions and results based on given medical cases and datasets. Higher scores indicate better academic performance. The experimental group's scores will be compared with the historical control group
次要结局
- Objective Knowledge Acquisition Rate(After the intervention (at the end of the course, approximately week 3))
- Perceived Usefulness and Technology Acceptance(After the intervention (at the end of the course, approximately week 3))
- AI Agent Engagement: Interaction Frequency(At the end of the course (approximately Week 3))
- AI Agent Engagement: Temporal Patterns(At the end of the course (approximately Week 3))
- AI Agent Engagement: Query Themes(At the end of the course (approximately Week 3))
研究者
Wei XIA, PhD
Associate Professor
Sun Yat-sen University