Evaluation of AI Support for Differential Diagnosis in Emergency Medicine.

The study is based on a prospective cohort of patients admitted through the emergency departments at Karolinska University Hospital in Huddinge and Solna. After completion of the hospital length of stay, excerpts from the medical record are collected according to a predefined variable list and used to create case descriptions corresponding to the information available at the initial assessment. Within this cohort, physicians will be randomized to evaluate patient cases either with or without AI support. The AI tool generates a structured list of likely differential diagnoses. Differential diagnoses proposed by physicians, AI alone, and physicians using AI support will be compared with the clinically established final diagnosis in the discharge summary. In a pilot material, the AI support has shown good agreement with final diagnoses and expert clinical assessment.

The results will be analyzed to assess AI's impact on physicians' diagnostic accuracy and decision-making in various scenarios and case presentations.

Given known challenges in diagnosing complex patients, the study is designed to account for variations in information quality and clinical complexity. Patient cases are presented in a standardized format, and it will be documented how much and what type of information is available in each case, such as symptoms, clinical findings, laboratory results, and disease progression. The analysis focuses on how this may affect the diagnostic suggestions. Difficult-to-diagnose cases are also included to clarify in which clinical situations the decision-support tool is useful, when it is not, and what informational requirements need to be met ahead of a future implementation study evaluating the impact of real-time AI support during the management of acutely ill patients.

There is a lack of large clinical studies that systematically evaluate how generative AI affects physicians' differential diagnostic performance in emergency medicine when used as a complement to clinical judgment. This project addresses this knowledge gap by examining the extent to which AI-generated differential diagnoses align with physicians' assessments and whether access to AI support can improve diagnostic precision.

The results will provide evidence on how AI-based decision support should be evaluated and potentially implemented within emergency care.