Prospective Multi-Center Evaluation of AI-Assisted Interpretation of Ultra-Widefield Retinal Images in a Multi-Reader Crossover Study
试验速览
- 阶段
- 不适用
- 状态
- 已完成
- 发起方
- 入组人数
- 462
- 试验地点
- 5
- 主要终点
- Sensitivity for retinal finding detection
研究概览
简要总结
The goal of this prospective observational study is to evaluate the impact of artificial intelligence (AI) assistance on clinician interpretation of ultra-widefield (UWF) retinal images.
The main questions it aims to answer are:
whether AI assistance improves the diagnostic performance of ophthalmologists in detecting retinal findings on UWF retinal images; whether AI assistance improves sensitivity, specificity, and inter-reader agreement across clinicians with different levels of experience.
Approximately 600 UWF retinal images prospectively collected from multiple ophthalmic centers in China will be included. Images will be independently annotated by expert retinal specialists to establish reference labels for retinal finding categories.
Four ophthalmologists with different levels of clinical experience, including one senior retinal specialist and three junior ophthalmologists, will participate in a crossover multi-reader study.
For each clinician, the dataset will be randomly divided into two equal subsets. During the first reading session, clinicians will evaluate one subset without AI assistance and the other subset with AI assistance. After a washout interval of at least two weeks, the reading conditions will be reversed in a second reading session with independently randomized image order.
Under the AI-assisted condition, clinicians will be provided with category-level AI prediction probabilities for retinal findings. No localization maps, heatmaps, segmentation overlays, or automated diagnostic recommendations will be displayed. Clinicians will retain full autonomy over final decisions.
Reader performance under AI-assisted and unaided conditions will be compared using expert reference annotations as the ground truth.
详细描述
This study is a prospective multi-center observational reader study designed to evaluate the impact of artificial intelligence (AI) assistance on clinician interpretation of ultra-widefield (UWF) retinal images.
Approximately 600 UWF retinal images will be prospectively collected from multiple ophthalmic centers in China. Images will be acquired using clinically routine UWF retinal imaging systems and will include a broad spectrum of retinal diseases and retinal findings encountered in real-world clinical practice.
All images will undergo independent expert annotation by retinal specialists to establish reference labels for retinal finding categories. These expert annotations will serve as the reference standard for subsequent performance evaluation.
Four ophthalmologists with different levels of clinical experience will participate in the reader study, including:
one senior retinal specialist with approximately five years of retinal clinical experience; three junior ophthalmologists with approximately two years of ophthalmology residency training.
研究设计
- 研究类型
- Observational
- 观察模型
- Other
- 时间视角
- Prospective
入排标准
- 年龄范围
- 18 Years 至 —（Adult, Older Adult）
- 性别
- All
- 接受健康志愿者
- 是
入选标准
- •Participants undergoing ultra-widefield retinal imaging at participating ophthalmic centers;
排除标准
- •Poor-quality or ungradable retinal images;
研究组 & 干预措施
AI-Assisted Interpretation
Clinicians interpret ultra-widefield retinal images with access to AI-generated category-level prediction probabilities for retinal findings.
干预措施: AI-Assisted Interpretation (Device)
Unaided Interpretation
Clinicians interpret ultra-widefield retinal images without AI assistance using routine retinal image interpretation alone.
干预措施: Unaided Interpretation (Device)
结局指标
主要结局
Sensitivity for retinal finding detection
时间窗: through study completion, an average of 2 months
Sensitivity of clinicians in detecting retinal finding categories under AI-assisted and unaided conditions using expert annotations as the reference standard.
Specificity for retinal finding detection
时间窗: through study completion, an average of 2 months
Specificity of clinicians in detecting retinal finding categories under AI-assisted and unaided conditions.
次要结局
- Area under the receiver operating characteristic curve (AUC)(through study completion, an average of 2 months)
- Inter-reader agreement(At study completion (up to 3 months))
- Diagnostic performance improvement among junior ophthalmologists(At study completion (up to 3 months))
研究者
XiujuChen
Investigator
Xiamen Ophthalmology Center Affiliated to Xiamen University