A Prospective, Cross-Sectional, Vignette-Based Observational Study Comparing Clinical Decision-Making Performance of Pediatriciansand AI Models
试验速览
- 阶段
- 不适用
- 状态
- 已完成
- 发起方
- 入组人数
- 30
- 试验地点
- 1
- 主要终点
- AI Interpretation Accuracy (%)
研究概览
简要总结
This study evaluates how well anonymized artificial-intelligence (AI) tools perform on standardized pediatric case vignettes and whether showing AI suggestions can improve clinicians' answers. About 30 board-certified/eligible pediatric specialists at a single hospital complete a one-time session. Participants are randomized to two groups. Group A (n≈15): physicians answer each vignette once. Group B (n≈15): physicians answer and rate confidence (1-10), then review anonymized suggestions from five different AI tools (tool names not shown) and may keep or change their answer; changes and confidence are recorded.
Primary focus: measure AI performance (diagnostic accuracy, medication-dosing accuracy, interpretation accuracy) overall and by difficulty tier, and record AI response time. Secondary focus: quantify how AI suggestions affect human performance (change in accuracy, direction of change, confidence shift, and time). No patients or biospecimens are involved; risks are minimal (time and possible discomfort with performance review). Findings may inform safe, evidence-based ways to use AI alongside clinicians in pediatrics.
研究设计
- 研究类型
- Observational
- 观察模型
- Cohort
- 时间视角
- Cross Sectional
入排标准
- 年龄范围
- 28 Years 至 40 Years（Adult）
- 性别
- All
- 接受健康志愿者
- 是
入选标准
- •Board-certified or board-eligible pediatric specialist (general pediatrics) (in the first 10 years of expertise)
- •Actively practicing at the participating institution/network at the time of enrollment.
- •Able and willing to complete all vignette items individually in a single session and to follow study instructions for the assigned cohort (direct answers or confidence rating + viewing anonymized AI suggestions).
- •Fluent in Turkish and able to use a computer interface.
- •Provides written informed consent.
排除标准
- •Pediatric subspecialist practice as primary role (e.g., cardiology, infectious diseases, neurology, neonatology, etc.), to maintain a homogeneous general pediatrics cohort.
- •Prior access to or participation in creating the study vignettes, answer keys, or scoring rubrics; direct involvement with the study team.
- •Inability to complete the session without external help or use of non-protocol resources (internet/AI tools) during answering (outside of anonymized AI suggestions shown by the system in Group 2).
- •Failure to complete ≥90% of items or major protocol deviation (e.g., discussion with others during the task).
- •Any condition judged by investigators to interfere with valid participation (e.g., severe time constraints, inability to provide consent).
结局指标
主要结局
AI Interpretation Accuracy (%)
时间窗: Day 1
Proportion of correct laboratory/imaging interpretations or appropriate next-test selections, per AI tool and pooled; stratified by difficulty tier. Unit: percent (0-100).
AI Diagnostic Accuracy (%)
时间窗: Day 1
Proportion of vignettes with a correct primary diagnosis produced by each anonymized AI tool and pooled across tools. Correctness is defined against a pre-specified reference answer key; results are also stratified by pre-defined difficulty tiers (easy/moderate/difficult/very difficult). Unit of measure: percent (0-100).
AI Medication-Dosing Accuracy (%)
时间窗: Day 1
Proportion of dose recommendations meeting pediatric standards (weight- or BSA-based ranges, route, frequency) per reference rubric, per AI tool and pooled; stratified by difficulty tier. Unit: percent (0-100).
次要结局
- Change in Physician Diagnostic Accuracy (percentage points) (Group 2 only)(Day 1: Baseline (pre-AI) and immediate Post-AI within the same session (0-15 min after baseline).)
- Confidence Shift (Δ on a 1-10 scale) (Group 2 only)(Day 1: Baseline (pre-AI) and immediate Post-AI within the same session (0-15 min after baseline).)
- Answer-Change Frequency (%) (Group 2 only)(Day 1)
- AI Response Time (seconds per vignette)(Day 1)
- Net Benefit Index of AI Exposure (percentage points) (Group 2 only)(Day 1)
研究者
Berker Okay
MD - Pediatrician (Principal Investigator)
Haseki Training and Research Hospital