A Multicentre Randomised Controlled Trial of LLM-Assisted Diagnostic Support in Patients With Suspected Rare or Diagnostically Unresolved Disease
Trial Snapshot
- Phase
- Not Applicable
- Status
- Not yet recruiting
- Enrollment
- 1,056
- Locations
- 13
- Primary Endpoint
- Rare Genetic Disease Diagnostic Yield
Study Overview
Brief Summary
A multicentre randomised controlled trial evaluating whether a rare-disease diagnostic large language model can improve diagnostic quality, efficiency, and health-economic outcomes for physicians managing patients with suspected rare or diagnostically unresolved disease.
Detailed Description
Rare disease patients commonly experience prolonged diagnostic odysseys rooted in limited rare disease recognition, phenotypic heterogeneity, and dispersed diagnostic clues. Diagnostic decision-support large language models may improve first-visit consultations by integrating prior records, generating structured analyses, and proposing candidate diagnoses, thereby shortening diagnostic pathways and improving appropriate genetic testing referral.
Participating physicians will provide care under both AI-assisted and standard diagnostic workflows. Eligible patients will be individually randomised to receive either AI-assisted diagnostic support or standard clinical practice.
In the intervention arm, physicians will have diagnostic support from AI when seeing patients. In the control arm, patients are seen under standard hospital workflow without any generative AI tools. Outcomes adjudicated by an independent Expert Committee blinded to arm assignment; adjudicators access no AI-generated materials.
A prospective within-trial economic evaluation will be conducted alongside the randomized trial. Healthcare resource use and costs associated with the diagnostic pathway will be collected.
Study Design
- Study Type
- Interventional
- Allocation
- Randomized
- Intervention Model
- Parallel
- Primary Purpose
- Diagnostic
- Masking
- Single (Outcomes Assessor)
Masking Description
Patient level: open label. Treating physicians: open label. Outcome adjudicators (independent expert committee) and statisticians: blinded. Adjudicators do not know arm assignment and are not shown any AI-generated or AI-attributed material; statistical analysis coded A/B until unblinding.
Eligibility Criteria
- Ages
- 0 Years to — (Child, Adult, Older Adult)
- Sex
- All
- Accepts Healthy Volunteers
- No
Inclusion Criteria
- •Patient Inclusion Criteria:
- •Any age. Legal guardian co-signs consent for minors or individuals lacking legal capacity.
- •Diagnostically unresolved or suspected rare disease, with at least one prior complete clinical evaluation at a secondary-level or higher institution yielding no confirmed explanatory diagnosis.
- •First presentation to the enrolling institution for the current condition, with no prior records in the institutional HIS or outpatient system.
- •No prior genetic testing related to the current condition; no results or reports available.
- •Written informed consent provided voluntarily by patient or legal guardian, with commitment and ability to complete structured follow-up.
Exclusion Criteria
- •Confirmed diagnosis (clinical, pathological, or molecular) explaining the primary symptoms.
- •Emergency presentation, critical illness, or any condition incompatible with trial participation.
- •Neither patient nor legally authorised proxy able to complete follow-up.
- •Concurrent enrollment in another interventional study with diagnostic accuracy or genetic testing yield as a primary endpoint.
- •Prior use of another AI system has already yielded a confirmed diagnosis for the current condition.
- •Physician Inclusion Criteria
- •Licensed physician in internal medicine, neurology, pediatrics, general medicine, rare disease, or a related specialty.
- •≥2 years of clinical practice; competent to manage rare disease patients; stratified into junior or senior tier.
- •Voluntary participation with written informed consent.
- •Physician Exclusion Criteria
- •No longer in clinical practice, or unable to fulfill required outpatient duties during the study period.
- •Unwilling to provide informed consent or to permit protocol-required collection of consultation and questionnaire data.
- •Currently enrolled in another AI-assisted clinical workflow, or expected to be unable to comply with the procedures.
Arms & Interventions
AI system
AI system will be used to provide diagnostic support during the encounter in addition to conventional clinical workflow. Use of other generative AI tools is prohibited.
Intervention: AI system (Other)
Standard of care
The physician conducts the encounter per standard hospital workflow using conventional clinical resources only. Use of any generative AI tool is prohibited.
Outcomes
Primary Outcomes
Rare Genetic Disease Diagnostic Yield
Time Frame: From the first visit to final reference diagnosis adjudication, an average of 8 weeks.
The proportion of all randomised patients who underwent genetic testing and had a clinically significant genetic variant (pathogenic or likely pathogenic) confirmed by an independent review committee blinded to arm assignment (denominator: all randomised patients). Variant pathogenicity classified per predefined ACMG/AMP guidelines.
Top-3 Candidate Diagnostic Accuracy
Time Frame: From the first visit to final reference diagnosis adjudication, an average of 8 weeks.
The proportion of patients whose physician-provided up to 3 candidate diagnoses at the first study colsultation include at least one concordant with the final reference diagnosis.
Molecular Diagnostic Yield
Time Frame: From the first visit to final reference diagnosis adjudication, an average of 8 weeks.
The proportion of all randomized patients who achieve a confirmed molecular diagnosis through the clinical diagnostic pathway. A molecular diagnosis will be counted when genetic testing recommended by the treating physician at the initial consultation yields a clinically relevant genetic finding confirmed by an independent genetics adjudication committee.
Overall Correct Diagnostic Yield
Time Frame: From the first visit to final reference diagnosis adjudication, an average of 8 weeks.
The proportion of all randomised patients whose clinical diagnosis by the end of follow-up is concordant with the blinded-adjudicated final reference diagnosis determined by an independent committee.
Secondary Outcomes
- Overall Diagnostic Yield(From the first visit to final reference diagnosis adjudication, an average of 8 weeks.)
- Overall Diagnostic Accuracy(From the first visit to final reference diagnosis adjudication, an average of 8 weeks.)
- Time to Diagnosis(From enrollment to the end of follow-up, up to 8 weeks.)
- Consultation Duration(Assessed at each consultation (day 1), within 1 day.)
- Cost to Diagnosis(From enrollment to the end of follow-up, up to 8 weeks.)
- Physician-Reported Experience(Assessed at each consultation (day 1), within 1 day.)
- Patient-Reported Experience(Assessed at each consultation (day 1), within 1 day.)
- Genetic Testing Referral Rate(Assessed at each consultation (day 1), within 1 day.)
- Positive Rate Among Referred Patients(From the first visit to final reference diagnosis adjudication, an average of 8 weeks.)
- Cost-Effectiveness Analysis(From the first visit to diagnosis or the end of follow-up, up to 8 weeks.)
- Overall Correct Diagnostic Yield(From the first visit to final reference diagnosis adjudication, an average of 8 weeks.)
- Physician Genetic Testing Recommendation Rate(At completion of the initial consultation (Day 1), within 1 day.)
- Appropriate Genetic Testing Recommendation Rate(From the initial consultation to genetic testing indication adjudication, approximately 8 weeks)
- Inappropriate Genetic Testing Recommendation Rate(From the initial consultation to genetic testing indication adjudication, approximately 8 weeks)
- Missed Genetic Testing Recommendation Rate(From the initial consultation to genetic testing indication adjudication, approximately 8 weeks)
- AI-Assisted Genetic Testing Recommendation Concordance(At completion of the AI-assisted consultation (Day 1))
- Candidate Diagnostic Accuracy(From the first visit to final reference diagnosis adjudication, an average of 8 weeks.)
- Molecular Diagnostic Yield(From the first visit to final reference diagnosis adjudication, an average of 8 weeks.)
- Time to a Correct Diagnosis(From enrollment to the end of follow-up, up to 8 weeks.)
- Duration of the Initial Physician Consultation(Assessed at each consultation (day 1), within 1 day.)
Investigators
Shuyang Zhang, MD, PhD
President of PUMCH
Peking Union Medical College Hospital