Al to Improve the Diagnosis of Rare Rheumatic Diseases

Not Applicable

Not yet recruiting

• Conditions: Rheumatic Diseases

• Registration Number: NCT06748170
• Lead Sponsor: Philipps University Marburg

Brief Summary

This trial aims to assess the impact of providing medical students with access to ChatGPT, a state-of-the-art large language model, in comparison to conventional diagnostic decision support tools, on their diagnostic accuracy for rare rheumatic diseases.

Detailed Description

Advanced artificial intelligence (AI) technologies, particularly large language models such as OpenAI's ChatGPT, hold significant potential for enhancing medical decision-making. While ChatGPT was not specifically designed for medical applications, it has shown utility in various healthcare scenarios, including answering patient inquiries, drafting medical documentation, and aiding consultations. Despite these advancements, its role in supporting diagnostic reasoning-especially among less experienced medical students-and for complex rare diseases remains underexplored.

Diagnostic reasoning is a multifaceted process that combines pattern recognition, knowledge synthesis, and probabilistic thinking. Tools like ChatGPT could potentially alleviate cognitive burden, enhance diagnostic accuracy, and ultimately accelerate the diagnosis for rare diseases. However, ChatGPT is not tailored for diagnostic reasoning and lacks comprehensive validation in this domain. Additionally, it is susceptible to generating misinformation or plausible-sounding but inaccurate responses, which may hinder rather than support clinical decision-making. Therefore, understanding how medical students utilize such AI tools is essential before they are integrated into educational or clinical workflows. This study will also assess a standardized prompt to facilitate ChatGPT usage and will give students direct access to enable a realistic scenario.

This study will investigate the impact of ChatGPT on the diagnostic accuracy of medical students when tackling cases of rare rheumatic diseases. Participants will be randomized into two groups: one with access to ChatGPT and one using conventional diagnostic tools. Each participant will analyze diagnostic cases by providing up to 5 differential diagnoses and and rating the diagnostic confidence. Independent reviewers, blinded to group allocation, will evaluate the accuracy and quality of their responses. This study hence aims to provide insights into the potential benefits and limitations of integrating AI tools like ChatGPT.

Study Timeline

• Status Verified: 2024-12
• Study Start: 2024-12
• Study First Submitted: 2024-12-16
• Study First Submitted QC: 2024-12-20
• Last Update Submitted: 2024-12-20
• Study First Posted: 2024-12-27
• Last Update Posted: 2024-12-27
• Primary Completion: 2025-03
• Study Completion: 2025-03

Recruitment & Eligibility

• Status: NOT_YET_RECRUITING
• Sex: All
• Target Recruitment: 68

Inclusion Criteria

• Medical students having started with clinical subjects (Internal medicine)

Exclusion Criteria

• Not being a medical student

Study & Design

• Study Type: INTERVENTIONAL
• Study Design: PARALLEL

Primary Outcome Measures

Name	Time	Method
Diagnostic accuracy of top diagnosis	during evaluation	Participants in each group will make at least one disease suggestion (top diagnosis) and up to a total of a maximum of 5 suggestions. Percentage of exact matches of the top suggestion with the actual diagnosis will be analyzed

Secondary Outcome Measures

Name	Time	Method
Diagnostic accuracy of top 5 suggestions	during evaluation	Participants in each group will make at least one disease suggestion (top diagnosis) and up to a total of a maximum of 5 suggestions. Percentage of exact matches with the actual diagnosis included in the top 5 suggestions will be analyzed
Diagnostic reasoning	during evaluation	For each case, participants will receive 1 point for each plausible diagnosis and 2 points for a completely correct response. The total scores will be compared between the randomized groups.
Diagnostic confidence	during evaluation	For each case participants will be asked for their diagnostic confidence (VAS 0-10). The mean score will be compared between groups.
Time spent for diagnosis	during evaluation	We will compare how much time (in seconds) participants spend per case between the two study arms.