Effect of Predictive Model on ED Physician Assessments of Patient Disposition
试验速览
- 阶段
- 不适用
- 状态
- 尚未招募
- 入组人数
- 10
- 主要终点
- Patient final disposition (admitted/discharged)
研究概览
简要总结
The goal of this study is to measure the impact of fairness-aware algorithms on physician predictions of ED patient admission. Using an experimentally validated machine learning model tuned for equitable outcomes, the investigators quantify the impact of model recommendations on ED physician assessments of admission risk in a silent, prospective study. The investigators survey ED physicians who are not currently caring for patients using live site data. To quantify the impact of the model on ED physician assessments of admission risk, the investigators collect physician assessments before and after consulting the (original or updated) model prediction.
The investigators measure ED physician adherence to model suggestions, along with the predictive accuracy and equity of downstream patient outcomes. The outcome of this study is an empirical measure of the extent to which fair ML models may influence admission decisions to mitigate health care disparities.
详细描述
Specific Aims/Objectives:
- Measure the effect of the sharing of a model prediction of admission on an attending physician's assessment of patient disposition within one hour from presentation at a tertiary academic pediatric hospital.
- Measure the effect of the sharing of a model prediction from a model tuned for equal subgroup performance on an attending physician's assessment of patient disposition within one hour from presentation at a tertiary academic pediatric hospital.
Background and Significance:
Machine learning (ML) models increasingly provide clinical decision support (CDS) to care teams to help prioritize individuals for specific care based on their predicted health needs and outcomes. AI/ML methods can have a particularly high impact on resource allocation in emergency departments (EDs) across the U.S., which have been described by the Institute for Medicine as "nearing the breaking point" of over-capacity. Unfortunately, models often perform poorly on disinvested subpopulations relative to the population as a whole. As a result, ML models may exacerbate downstream health disparities by under-performing on marginalized patient subpopulations, especially when models are expanded to multiple care centers and or used without subgroup monitoring for long periods of time.
Many prediction models have been developed in recent years to predict patient disposition from the ED, including a prediction tool developed by our group and currently in piloting stages at Boston Children's Hospital, South Shore Hospital, and Children's Hospital of Los-Angeles. Our prediction tool, the Predictor of Patient Placement (POPP) provides an accurate, real-time likelihood of admission based on data available in the electronic health record at the time of the visit. Advance notice of likely admissions can have an important impact on ED waiting and boarding times with the potential to improve flow and patient satisfaction.
研究设计
- 研究类型
- Interventional
- 分配方式
- Randomized
- 干预模型
- Sequential
- 主要目的
- Other
- 盲法
- Triple (Participant, Care Provider, Investigator)
盲法说明
We conduct a randomized, double-blind, controlled before-after (CBA) study of board-certified ED attending physicians not currently caring for patients in the BCH ED over a period of six to twelve weeks. In this experiment, the "treatment" consists of an ML recommendation provided to the ED physicians, who predict admission decisions for individual patients before and after receiving it. The "control" surveys receive the original POPP model recommendation, and "treatment" surveys receive a "fairness-aware" model, determined in prior work to mitigate biases in performance with respect to patient demographics
入排标准
- 年龄范围
- 18 Years 至 65 Years（Adult, Older Adult）
- 性别
- All
- 接受健康志愿者
- 是
入选标准
- •Board certified emergency department attending physicians currently employed by Boston Children's Hospital
排除标准
- •Physicians are excluded from completely surveys for patients who they are currently caring for
研究组 & 干预措施
Physician assessment before intervention
No intervention. Physician is surveyed to provide their assessment of patient disposition.
Physician assessment after baseline model intervention
Physician is shown a baseline model recommendation for patient disposition including description of factors driving the model prediction.
干预措施: Baseline model (Diagnostic Test)
Physician assessment after fairness-aware model intervention
Physician is shown a model recommendation form a model tuned for subgroup performance for patient disposition including description of factors driving the model prediction.
干预措施: Fairness-aware model (Diagnostic Test)
结局指标
主要结局
Patient final disposition (admitted/discharged)
时间窗: Within 24 hours of survey
The final disposition of the patient, whether admitted to an inpatient service or discharged
Physician-assessed ED disposition (likelihood of admission)
时间窗: Within 24 hours of survey
The primary outcome is physician-assessed ED disposition (categorized as admission or discharge), before and after viewing a model prediction, compared to final disposition of patient
次要结局
- Model-assessed ED disposition(Within 24 hours of survey)
研究者
William La Cava
Assistant Professor
Boston Children's Hospital