A Multicentre, Randomised Diagnostic Accuracy Study Evaluating AI Assisted Diagnosis of Rare Diseases

Rare diseases collectively affect approximately 300 million individuals worldwide. This prolonged diagnostic delay is attributable in large part to the breadth of over 7,000 recognized rare conditions, which far exceeds the clinical exposure of any individual physician. A rare disease-specific diagnostic AI was developed by Peking Union Medical College Hospital (PUMCH), supporting differential diagnosis generation, clinical workup planning, and genomic variant interpretation. A balanced crossover design ensures that each enrolled physician serves as their own control, substantially reducing confounding from inter-reader variability in baseline diagnostic competency. Within each physician, cases are randomly assigned at the case level to either the AI-assisted or unassisted condition, such that each physician reads a subset of cases with AI assistance and the remaining cases without. This within-reader, case-level randomization eliminates the need for a washout period and directly controls for inter-reader differences in baseline diagnostic competency. All cases are collected from real-world clinical settings with independently confirmed gold-standard diagnoses and span a pre-specified spectrum of rare and non-rare disease categories, reflecting the differential diagnostic challenge encountered in routine clinical practice, to ensure diagnostic breadth and clinical representativeness. Physician seniority (junior vs. senior) is incorporated as a pre-specified stratification and subgroup analysis variable. Diagnostic outputs are evaluated by an independent Expert Adjudication Committee, blinded to the assistance condition, using standardized scoring criteria established prior to data collection.