Evaluation of a Retrieval Augmented Large Language Model as a Diagnostic Copilot in Rheumatology

Advanced AI, particularly large language models, shows promise for enhancing clinical reasoning, yet most systems such as ChatGPT are not certified as medical products. Prof. Valmed is a clinical decision support medical product designed to assist physicians in diagnostic decision making. Given frequent referral problems and diagnostic delays in rheumatology, evaluating such support is highly relevant for clinical workflows.

This randomized controlled trial will test whether access to Prof. Valmed improves physicians' diagnostic performance in cases of suspected rheumatic disease compared with conventional decision support. Participants will be randomized to either use Prof. Valmed or rely on conventional tools while working through standardized clinical cases. For each case, participants will submit up to three differential diagnoses and a confidence rating. Independent reviewers, blinded to group allocation, will adjudicate accuracy. Findings will clarify the benefits and limitations of integrating Prof. Valmed into routine practice.