The Effects of a Large Language Model on Clinical Questioning Skills
- Conditions
- GlaucomaCataractKeratitisConjunctivitisDiabetic Retinopathy
- Interventions
- Device: "Digital twin patient"Behavioral: Interaction with real patients
- Registration Number
- NCT06229379
- Lead Sponsor
- Sun Yat-sen University
- Brief Summary
The researchers have used the ophthalmology textbook, clinical guideline consensus, the Internet conversation data and knowledge base of Zhongshan Ophthalmology Center in the early stage, combined with artificial feedback reinforcement learning and other techniques to fine-tune and train the LLM, and developed "Digital Twin Patient", a localized large language model that has the ability to answer ophthalmology-related medical questions, and also constructed a combination of automated model evaluation and manual evaluation by medical experts. The evaluation system combining automated model evaluation and manual evaluation by medical experts was constructed at the same time.
This project intends to integrate "Digital Twin Patient" into undergraduate ophthalmology apprenticeship, simulate the consultation process of real patients through the online interaction between students and "Digital Twin Patient", explore the effect of "Digital Twin Patient" consultation teaching, provide emerging technology tools for guiding medical students to actively learn a variety of ophthalmology cases, cultivate clinical thinking, and provide the possibility of creating a new mode of intelligent teaching.
- Detailed Description
At present, the main form of clinical questioning skills teaching is to let undergraduates who participate in the apprenticeship first learn the characteristics and diagnostic points of cases, and then practice questioning on real patients in the wards. However, due to the large number of trainee students, it is difficult to meet the teaching demand in terms of the number of cases available for questioning and the richness of disease types under the current teaching mode. Therefore, it is necessary to utilize new intelligent technologies and create a new model of questioning skills teaching to improve teaching efficiency and enhance students' clinical thinking.
Large-scale language modeling (LLM) is a deep learning technology that can learn knowledge from a large amount of text, and AI chatbots such as ChatGPT are a typical example of its application. AI chatbots are characterized by anthropomorphic comprehension and diversified natural language generation abilities in different contexts, and have been initially applied in the medical field, such as passing the U.S. Medical Licensing Examination, assisting in ophthalmic history documentation and answering ophthalmic questions. However, it has been found that although LLM has fair modeling performance in general medical knowledge, it still needs to be improved in the area of specialty diseases. Based on this, the researcher's team has used the ophthalmology textbook, clinical guideline consensus, the Internet conversation data and knowledge base of Zhongshan Ophthalmology Center in the early stage, combined with artificial feedback reinforcement learning and other techniques to fine-tune and train the LLM, and developed "Digital Twin Patient", a localized large language model that has the ability to answer ophthalmology-related medical questions, and also constructed a combination of automated model evaluation and manual evaluation by medical experts. The evaluation system combining automated model evaluation and manual evaluation by medical experts was constructed at the same time.
This project intends to integrate "Digital Twin Patient" into undergraduate ophthalmology apprenticeship, simulate the consultation process of real patients through the online interaction between students and "Digital Twin Patient", explore the effect of "Digital Twin Patient" consultation teaching, provide emerging technology tools for guiding medical students to actively learn a variety of ophthalmology cases, cultivate clinical thinking, and provide the possibility of creating a new mode of intelligent teaching.
Recruitment & Eligibility
- Status
- COMPLETED
- Sex
- All
- Target Recruitment
- 84
- All undergraduate students from Sun Yat-sen University who participate in the ophthalmological internship.
- Students who refuse to sign informed consent.
Study & Design
- Study Type
- INTERVENTIONAL
- Study Design
- PARALLEL
- Arm && Interventions
Group Intervention Description "Digital twin patient" + Real patient "Digital twin patient" The test was completed in 5 days: the students in the "Digital twin patient" + Real patient group were trained in history taking using a "digital twin patient" on Monday, and then took a 15-minute clinical questioning exam using the "digital twin patient" on Tuesday. After that, they were trained in history taking using a real patient on Wednesday, and then took a 15-minute clinical questioning exam using a "digital twin patient" on Thursday. Finally, they were tested regards clinical questioning skills using a real patient on Friday. "Digital twin patient" + Real patient Interaction with real patients The test was completed in 5 days: the students in the "Digital twin patient" + Real patient group were trained in history taking using a "digital twin patient" on Monday, and then took a 15-minute clinical questioning exam using the "digital twin patient" on Tuesday. After that, they were trained in history taking using a real patient on Wednesday, and then took a 15-minute clinical questioning exam using a "digital twin patient" on Thursday. Finally, they were tested regards clinical questioning skills using a real patient on Friday. Real patient + "Digital twin patient" Interaction with real patients The test was completed in 5 days: the students in the Real patient + "Digital twin patient" group were trained in history taking using a real patient on Monday, and then took a 15-minute clinical questioning exam using the "digital twin patient" on Tuesday. After that, they were trained in history taking using a "digital twin patient" on Wednesday, and then took a 15-minute clinical questioning exam using a "digital twin patient" on Thursday. Finally, they were tested regards clinical questioning skills using a real patient on Friday. Real patient + "Digital twin patient" "Digital twin patient" The test was completed in 5 days: the students in the Real patient + "Digital twin patient" group were trained in history taking using a real patient on Monday, and then took a 15-minute clinical questioning exam using the "digital twin patient" on Tuesday. After that, they were trained in history taking using a "digital twin patient" on Wednesday, and then took a 15-minute clinical questioning exam using a "digital twin patient" on Thursday. Finally, they were tested regards clinical questioning skills using a real patient on Friday.
- Primary Outcome Measures
Name Time Method Students' scores in the medical history acquisition exam Weekly during this study (up to 10 months)
- Secondary Outcome Measures
Name Time Method
Trial Locations
- Locations (1)
Zhongshan Ophthalmic Center, Sun Yat-sen Univerisity
🇨🇳Guangzhou, Guangdong, China