Does AI Make Clinicians More Appropriately Confident? A Randomized Study in Preterm Birth Prediction
试验速览
- 阶段
- 不适用
- 状态
- 招募中
- 入组人数
- 125
- 试验地点
- 18
- 主要终点
- Clinician diagnostic calibration (accuracy-confidence alignment) after AI exposure.
研究概览
简要总结
The goal of this randomized questionnaire-based study is to evaluate how different presentations of artificial intelligence (AI) decision support influence clinical judgment among medical doctors working in obstetrics and gynecology when assessing the risk of spontaneous preterm birth using clinical case vignettes with cervical ultrasound images. The study specifically compares two AI presentation formats: a binary classification (preterm vs term birth) and an individualized risk estimate of preterm birth.
The main questions it aims to answer are:
- Which AI presentation format leads to better alignment between clinicians' confidence and decision accuracy (diagnostic calibration)?
- Do different AI presentation formats lead to helpful or harmful changes in clinical decisions?
Participants will complete an online questionnaire in which they review clinical cases, make diagnostic and management decisions, rate their diagnostic confidence before and after seeing the AI output, and report their trust in the AI.
研究设计
- 研究类型
- Interventional
- 分配方式
- Randomized
- 干预模型
- Parallel
- 主要目的
- Other
- 盲法
- Single (Participant)
盲法说明
Participants (clinicians) are blinded to randomized allocation and are unaware that different versions of the AI decision support are being compared. They view only the AI output presented within their assigned condition. No independent outcome assessors are involved. Outcomes are derived using pre-specified, objective scoring rules.
入排标准
- 性别
- All
- 接受健康志愿者
- 是
入选标准
- •Medical doctors currently working in or training within the field of obstetrics and gynecology.
- •Experience performing transvaginal cervical ultrasound examinations.
排除标准
- •- No prior experience performing transvaginal cervical ultrasound examinations.
研究组 & 干预措施
AI prediction
The participants receive a binary AI prediction (preterm or term birth)
干预措施: AI prediction (binary) (Behavioral)
AI risk estimate
The participants receive an AI risk estimate of preterm birth (%)
干预措施: AI risk estimate (%) (Behavioral)
结局指标
主要结局
Clinician diagnostic calibration (accuracy-confidence alignment) after AI exposure.
时间窗: Immediately after AI exposure during a single questionnaire session (approximately 20 minutes).
Agreement between post-AI decision correctness (0/1) and post-AI confidence rating (0-10) will be quantified using the Brier score. Confidence will be rescaled to 0-1 and squared differences between confidence and correctness will be averaged across cases to produce a participant-level score. Lower scores indicate better diagnostic calibration. Results will be compared between randomized arms.
次要结局
- Helpful switch rate and harmful switch rate.(Baseline (pre-AI) and immediately after AI exposure during a single questionnaire session (approximately 20 minutes).)
- Change in decision accuracy, confidence, and diagnostic calibration from pre-AI to post-AI.(Baseline (pre-AI) and immediately after AI exposure during a single questionnaire session (approximately 20 minutes).)
- Association between self-rated trust in AI and behavioral reliance on AI.(Immediately after AI exposure during a single questionnaire session (approximately 20 minutes).)
- Follow-up cervical ultrasound planning.(Baseline (pre-AI) and immediately after AI exposure during a single questionnaire session (approximately 20 minutes).)
研究者
Emilie Pi Fogtmann Sejer
Principal Investigator
Rigshospitalet, Denmark