Impact of Supplementary Access to a Fine-Tuned Medical Large Language Model (MetaGP-Edu) on Learning Outcomes in Undergraduate Medicine Education

ackground and Rationale: The effective teaching of Internal Medicine, a cornerstone of undergraduate medical education, presents significant pedagogical challenges due to the breadth and complexity of the subject matter and the large student cohorts typically enrolled in major academic medical centers. While traditional methods like lectures and textbook readings are essential, there is a recognized need for innovative approaches that can better support the development of clinical reasoning, facilitate deeper engagement with complex case material, and offer more personalized learning opportunities at scale. Artificial intelligence (AI), particularly advanced large language models (LLMs) trained on domain-specific knowledge, holds considerable potential as an educational technology to address these needs. MetaGP-Edu, a proprietary generative foundation model fine-tuned specifically for medical education using pedagogical datasets, was developed to explore this potential by serving as a supplementary learning resource. Evaluating the real-world impact of integrating such tools into established curricula is crucial for evidence-informed educational practice.

Objectives: The primary objective of this study is to retrospectively evaluate the association between the availability of the MetaGP-Edu AI tool as a supplementary learning resource and overall student academic performance in the core Internal Medicine curriculum.

Study Design: This investigation employs a multi-center, retrospective cohort study design. Routinely collected academic data from several major medical schools in China over multiple consecutive academic years will be analyzed. This approach allows for the comparison of student cohorts based on their historical exposure to different educational resource environments (with vs. without MetaGP-Edu access) within real-world academic settings.

Setting and Participants: The study encompasses data from undergraduate medical students enrolled in the mandatory Internal Medicine course at several large, academically affiliated medical institutions in China. Participants include students who completed the course across a span of academic years covering the period before and after the introduction of MetaGP-Edu (approximately Fall 2022). Inclusion is based on the availability of complete academic records for the Internal Medicine course during the specified study period.

Exposure/Intervention and Comparator: The study compares two main cohorts defined by the timing of MetaGP-Edu availability: