Effects of Expert Arbitration on Clinical Outcomes When Disputes Over Diagnosis Arise Between Physicians and Their Artificial Intelligence Counterparts: a Randomized, Multicenter Trial in Pediatric Outpatients

Based on the historical clinical data of more than 1 million pediatric outpatients in the Guangzhou Women and Children's Medical Center, an AI diagnostic framework has recently been developed for common pediatric diseases [Liang H et al. evaluation and accurate diagnosis of pediatric disease using artificial intelligence. Nat Med. 2019;25(3):433-8]. This AI framework utilizes predefined schema to extract informative clinical data from free text and reaches clinical diagnoses by hypothetico-deductive reasoning. In internal validation, the AI system showed accuracy rates ranging from 0.85 for gastrointestinal disease to 0.98 for neuropsychiatric disorders, suggesting that it might be a promising assisting diagnostic tool in clinical practice. However, there is a lack of evidence-based strategy on how to handle the scenarios where the AI-based diagnosis and the diagnosis made by pediatricians are discordant. It is legitimate to assume that diseases with discordant diagnoses present more similar clinical features; in this case it is necessary to introduce an extra arbitrator for differential and decisive diagnosis. Therefore, we conduct this randomized controlled trial to: 1) compare the accuracy of the two diagnostic modes in a real-world clinical setting where the AI-based diagnosis and the diagnosis made by pediatricians are discordant by introducing an expert arbitrator; and 2) look further into the change of clinical outcomes (hospital revisit and hospitalization in the next 3 months after initial visit; average total outpatient cost) due to introduction of the expert arbitrator. Please note that although the aforementioned AI framework was designed for diagnosis of a wide range of diseases, this clinical trial is limited to outpatients encountered in three specialty clinics, i.e. respirology, gastroenterology, and genito-urology. The reason for this selection is that the discordant diagnoses are assumed to be more common for these two specialties according to the internal validation result.