Effectiveness of AI-Assisted Antibiotic Prescribing: A Randomized Controlled Trial

Inappropriate antibiotic prescribing remains one of the most consequential drivers of antimicrobial resistance globally, with particular urgency in low- and middle-income countries, where regulatory enforcement, antibiogram coverage, and specialist infectious-disease support are limited. Recent advances in Large Language Models (LLMs) have shown promise in supporting clinical decision-making. LLMs have been explored as tools for synthesizing antibiotic recommendations and flagging spectrum mismatches. Yet LLMs may produce confident but incorrect recommendations, and their use may reduce the quality of physicians' own clinical reasoning through over-reliance. Whether this promise translates into measurably better antibiotic prescribing behavior among practising physicians, measured across the full spectrum of selection, dosing, and clinical reasoning, has not been rigorously evaluated in a randomized, controlled setting.

This study evaluates whether access to an LLM affects the Antibiotic Prescribing Appropriateness Score (APAS) among licensed physicians in Pakistan. Secondary objectives include assessing whether the intervention improves antibiotic selection and spectrum-matching, whether the effect differs by physician clinical experience, and whether LLM access affects the time physicians spend per vignette.

This study will be an assessor-blind, randomized controlled trial with two arms. Participants will be randomly assigned to either the intervention or control arm in a 1:1 ratio. Outcome assessors (those scoring the APAS responses) will be blinded to participants' group assignments and will evaluate vignette responses without knowledge of which arm the responding physician was in. Each participant is expected to complete 6-10 vignettes in a single, proctored, 75-minute session. The order of vignettes will be randomized independently for each participant to control for order effects. The time spent on each vignette will be automatically recorded.

In the intervention arm, participants will have access to ChatGPT in addition to conventional diagnostic and reference resources (UpToDate, PubMed, Google search with AI features disabled). In the control arm, participants will use conventional resources only (UpToDate, PubMed, Google Search with AI features disabled). Both arms will evaluate the same set of clinical vignettes.

Participants will be presented with clinical vignettes covering common infectious disease scenarios requiring antibiotic prescribing decisions. Vignettes will be sourced and modified based on clinically realistic cases relevant to the target population, and will follow a standardized format, including the patient's chief complaint and history of present illness, and physical examination findings.