xrAI - Improving Quality and Efficiency in Chest Radiograph Interpretation by Radiologists
试验速览
- 阶段
- 不适用
- 状态
- 撤回
- 发起方
- 主要终点
- Number of abnormalities identified divided by number of total of images analyzed (accuracy)
研究概览
简要总结
xrAI (pronounced "X-ray") serves as a clinical assistance tool for trained clinical professionals who are interpreting chest radiographs. The tool is designed as a quality control and adjunct, limited, clinical decision support tool, and does not replace the role of clinical professionals. It highlights areas on chest radiographs for review by an interpreting clinician.
The objective of this study is to utilize machine learning and artificial intelligence algorithms (xrAI) to improve the quality and efficiency in the interpretation of chest radiographs by radiologists.
The hypothesis is that the addition of xrAI's analysis will reduce inter-observer variability in the interpretation of chest radiographs and increase participants' sensitivity, recall, and accuracy in pulmonary abnormality screening.
详细描述
To investigate the effect of xrAI for radiologists that interpret chest radiographs as part of their daily responsibilities, the investigators have designed a randomized control trial.
The pulmonary abnormalities detected by xrAI and included in the definition of abnormal are as follows: any linear scar or fibrosis, atelectasis, consolidation, abscess or cavity, nodule, pleural effusion, severe cases of emphysema and COPD (mild cases with hyperinflation but not significant emphysema are not flagged), and pneumothorax.
To assess the causal effect of xrAI the investigators randomly assign 10 to 14 radiologists to either treatment (x-ray images processed by xrAI) or control (no xrAI processing) groups. Participants will only review images once. Each participant will perform 500 radiograph interpretations in total.
Participants in the control group will be asked to interpret the same 500 images without xrAI's analysis.
To increase the precision of the estimate and better investigate potential differences between clinical professionals, investigators block randomize the assignment of treatment or control group.
研究设计
- 研究类型
- Interventional
- 分配方式
- Randomized
- 干预模型
- Parallel
- 主要目的
- Diagnostic
- 盲法
- Single (Participant)
入排标准
- 性别
- All
- 接受健康志愿者
- 是
入选标准
- •Radiologist currently practicing at a Pureform Radiology clinic in Calgary, Canada.
排除标准
- •Radiologists not currently practicing at a Pureform Radiology clinic in Calgary, Canada.
- •Physicians currently practicing at a Pureform Radiology clinic in Calgary, Canada, but that are not radiologists by training.
结局指标
主要结局
Number of abnormalities identified divided by number of total of images analyzed (accuracy)
时间窗: Time needed to analyze 500 images. Participants will be asked to completed the exercise within 2 weeks.
Accuracy is defined as the ratio of the images where the physician's prediction matched the labels of the dataset. Accuracy= (TP+FP) / (TP+FP+TN+FN) TP (true positives) = cases interpreted as abnormal that are abnormal; FP (false positives) = cases wrongly interpreted to be abnormal; TN (true negatives) = cases correctly interpreted to be normal; FN (false negatives) = abnormal cases wrongly interpreted as normal.
Number of true abnormalities identified divided by the total of abnormalities identified (precision)
时间窗: Time needed to analyze 500 images. Participants will be asked to completed the exercise within 2 weeks.
Precision is defined as the probability of a radiograph being abnormal if a physician makes the determination that it is abnormal. Precision= TP / (TP+FP) TP (true positives) = cases interpreted as abnormal that are abnormal; FP (false positives) = cases wrongly interpreted to be abnormal; TN (true negatives) = cases correctly interpreted to be normal; FN (false negatives) = abnormal cases wrongly interpreted as normal.
Number of true abnormalities identified divided by the sum of true abnormalities identified and abnormalities missed (recall)
时间窗: Time needed to analyze 500 images. Participants will be asked to completed the exercise within 2 weeks.
Recall is defined as the probability of a physician catching an abnormality in an image if one exists (based on the labels of the dataset). Recall= TP / (TP+FN) TP (true positives) = cases interpreted as abnormal that are abnormal; FP (false positives) = cases wrongly interpreted to be abnormal; TN (true negatives) = cases correctly interpreted to be normal; FN (false negatives) = abnormal cases wrongly interpreted as normal.
次要结局
- Mean of radiologist recall (as defined in outcome 3)(Time needed to analyze 500 images. Participants will be asked to completed the exercise within 2 weeks.)
- Mean of radiologist accuracy (as defined in outcome 1)(Time needed to analyze 500 images. Participants will be asked to completed the exercise within 2 weeks.)
- Mean of radiologist precision (as defined in outcome 2)(Time needed to analyze 500 images. Participants will be asked to completed the exercise within 2 weeks.)