DISCORD-Dx as a Cognitive Strategy for Calibrated Reliance on AI in Diagnosis
试验速览
- 阶段
- 不适用
- 状态
- 已完成
- 发起方
- 入组人数
- 46
- 试验地点
- 1
- 主要终点
- Proportion of Case-Level Decisions Demonstrating Appropriate Reliance on AI Recommendations
研究概览
简要总结
This randomized study evaluated whether a structured reasoning strategy called DISCORD-Dx could help specialist doctors use artificial intelligence (AI) recommendations more safely during diagnostic decision-making. The strategy was designed to help specialists benefit from correct AI advice while resisting plausible but incorrect AI recommendations.
Forty-six consultant specialists in pathology and diagnostic radiology from two hospitals in Bahawalpur, Pakistan, were randomly assigned to receive either DISCORD-Dx training or time-matched conventional AI-literacy training. During assessment, participants first recorded and locked their own diagnosis before seeing a standardized AI recommendation. They then reviewed the AI advice and entered a final diagnosis. The AI recommendations included both correct recommendations and deliberately generated plausible errors that had been independently reviewed by specialists.
The main outcome was appropriate reliance on AI, defined as following correct AI advice or resisting erroneous AI advice immediately after training. Other outcomes included harmful switching from a correct diagnosis to an incorrect diagnosis after erroneous AI advice, over-reliance, under-reliance, final diagnostic accuracy, decision time, and appropriate reliance at 8 weeks. No participant interacted with a live AI system, and study responses did not affect patient care.
详细描述
DISCORD-Dx is a seven-step cognitive strategy designed to support calibrated reliance on AI during diagnostic decision-making: Diagnose independently; Inspect the AI recommendation; Substantiate the evidence; Classify the disagreement; Override, modify, or accept; Review the outcome; and Demonstrate transfer.
Participants were consultant specialists with independent diagnostic responsibility in chemical pathology, hematology, microbiology, histopathology, or diagnostic radiology. Before randomization, participants completed four specialty-specific cases without AI. They were then randomized 1:1 to DISCORD-Dx training or an active control consisting of conventional AI-literacy training. Both groups received the same 20-minute orientation on AI limitations, confident error, verification, privacy, bias, and professional oversight, followed by a 40-minute group-specific training module.
Immediately after training, participants completed eight unseen specialty-specific cases, four containing correct AI recommendations and four containing deliberately erroneous but plausible AI recommendations. For each case, participants recorded an independent diagnosis before viewing the AI recommendation and then entered their final decision after reviewing the AI advice. At 8 weeks, participants completed six additional cases without refresher training. All AI recommendations were frozen and independently expert-reviewed before participant exposure, and participants did not interact with a live AI system.
研究设计
- 研究类型
- 干预性
- 分配方式
- 随机
- 干预模型
- 平行分组
- 主要目的
- 其他
- 盲法
- 单盲 (结局评估者)
盲法说明
Outcome assessors were blinded to treatment allocation. The primary analyst also remained masked to treatment codes through response scoring and primary-model diagnostic checks. Participants and facilitators were not blinded because the training content differed between groups.
入排标准
- 性别
- All
- 接受健康志愿者
- 是
入选标准
- Practicing consultant specialist with current independent diagnostic responsibility.
- Specialty in chemical pathology, hematology, microbiology, histopathology, or diagnostic radiology.
排除标准
- Trainee status.
- Absence of independent reporting responsibility.
- Direct involvement in constructing or validating the participant's specialty case bank.
- Prior access to assessment cases or scoring keys.
- Inability to complete the digital assessment.
研究组 & 干预措施
DISCORD-Dx Training
Participants received a 20-minute standardized orientation on AI capabilities and limitations, confident error, verification, privacy, bias, and professional oversight, followed by a 40-minute structured DISCORD-Dx training module. The module covered seven steps: Diagnose independently; Inspect AI advice; Substantiate evidence; Classify disagreement; Override, modify, or accept; Review outcome; and Demonstrate transfer.
干预措施: DISCORD-Dx Training (Behavioral)
Conventional AI-Literacy Training
Participants received the same 20-minute standardized orientation on AI capabilities and limitations, confident error, verification, privacy, bias, and professional oversight, followed by 40 minutes of conventional AI-literacy reinforcement and matched case discussion without the DISCORD-Dx mnemonic, disagreement taxonomy, or explicit accept/modify/override sequence.
干预措施: Conventional AI-Literacy Training (Behavioral)
结局指标
主要结局
Proportion of Case-Level Decisions Demonstrating Appropriate Reliance on AI Recommendations
时间窗: Immediately after training, during the 8-case post-training assessment
Appropriate reliance was assessed at the case level and defined as following a correct AI recommendation or resisting an erroneous AI recommendation. Each participant completed eight unseen specialty-specific cases during the immediate assessment, including four cases with correct AI recommendations and four with erroneous AI recommendations. The outcome was expressed as the proportion of case-level decisions meeting the definition of appropriate reliance.
次要结局
- Proportion of Eligible Erroneous-AI Case Decisions With Harmful Switching(Immediately after training, during the 8-case post-training assessment)
- Proportion of Erroneous-AI Case Decisions Demonstrating Over-Reliance(Immediately after training, during the 8-case post-training assessment)
- Proportion of Correct-AI Case Decisions Demonstrating Under-Reliance(Immediately after training, during the 8-case post-training assessment)
- Proportion of Eligible Correct-AI Case Decisions With Beneficial Correction(Immediately after training, during the 8-case post-training assessment)
- Final Diagnostic Accuracy(Immediately after training, during the 8-case post-training assessment)
- Decision Time(Immediately after training, during the 8-case post-training assessment)
- Appropriate Reliance on AI Recommendations at 8 Weeks(8 weeks after training (target window ±7 days))
研究者
Sara Reza
Associate Professor
Quaid-e-Azam Medical College
研究点 (1)
标识符
- NCT 编号
- NCT07844889
- 其他研究编号
- QAMC-DISCORD-Dx-2026
日期
- 首次提交
- (15天前)
- 首次发布
- (前天)
- 主要完成日期
- (2个月前)
- 研究完成日期
- (28天前)
- 最近核实
- (29天前)
- 最近更新
- (前天)
监管与共享
- FDA 监管药物
- 否
- FDA 监管器械
- 否
- 个体参与者数据共享计划
- 是
- 是否有结果
- 否
Deidentified individual participant data underlying the published results will be shared, including participant-level data and eligible case-level data used in the primary, secondary, and exploratory analyses. Data will be shared without direct personal identifiers and will be accompanied by a data dictionary and relevant analysis documentation.