AI Assisted Detection of Chest X-Rays
- Conditions
- AtelectasisPulmonary CalcificationCardiomegalyPulmonary Nodules, MultiplePneumothorax; AcutePleural EffusionPneumothoraxPulmonary ConsolidationPneumoperitoneumPulmonary Nodules, Solitary
- Interventions
- Other: Cases readingOther: Ground truthing
- Registration Number
- NCT06075836
- Lead Sponsor
- Oxford University Hospitals NHS Trust
- Brief Summary
This study has been added as a sub study to the Simulation Training for Emergency Department Imaging 2 study (ClinicalTrials.gov ID NCT05427838).
The Lunit INSIGHT CXR is a validation study that aims to assess the utility of an Artificial Intelligence-based (AI) chest X-ray (CXR) interpretation tool in assisting the diagnostic accuracy, speed, and confidence of a varied group of healthcare professionals. The study will be conducted using 500 retrospectively collected inpatient and emergency department CXRs from two United Kingdom (UK) hospital trusts. Two fellowship trained thoracic radiologists will independently review all studies to establish the ground truth reference standard. The Lunit INSIGHT CXR tool will be used to analyze each CXR, and its performance will be measured against the expert readers. The study will evaluate the utility of the algorithm in improving reader accuracy and confidence as measured by sensitivity, specificity, positive predictive value, and negative predictive value. The study will measure the performance of the algorithm against ten abnormal findings, including pulmonary nodules/mass, consolidation, pneumothorax, atelectasis, calcification, cardiomegaly, fibrosis, mediastinal widening, pleural effusion, and pneumoperitoneum. The study will involve readers from various clinical professional groups with and without the assistance of Lunit INSIGHT CXR. The study will provide evidence on the impact of AI algorithms in assisting healthcare professionals such as emergency medicine and general medicine physicians who regularly review images in their daily practice.
- Detailed Description
Not available
Recruitment & Eligibility
- Status
- ACTIVE_NOT_RECRUITING
- Sex
- All
- Target Recruitment
- 33
- General radiologists/radiographers/physicians who review CXRs as part of their routine clinical practice
- Thoracic radiologists
- Non-radiology physicians with previous formal postgraduate CXR reporting training.
- Non-radiology physicians with previous career in radiology, respiratory medicine or thoracic surgery to registrar or consultant level
Study & Design
- Study Type
- OBSERVATIONAL
- Study Design
- Not specified
- Arm && Interventions
Group Intervention Description Readers/Participants Cases reading Reader Selection: 30 readers will be selected from the following five clinical specialty groups: * emergency medicine (ED) * adult intensive care (ICU) * adult general medicine (AGM) * radiographers (Rad) * general radiologists Each specialty group consists of 6 members of ranked seniority. For the physicians this consists of: * Two 'Juniors' (Foundation Year 1 - Specialty Training 2 years) * Two 'Middle Grades' (Registrar from Specialty Training 3 to 6 years) * Two Consultants For the radiographers, this consists of: * Two 'Junior/Newly qualified radiographers' (up to 18 months experience post qualification) * Two 'Mid-experience radiographers' (approx. 3 years' experience) * Two 'Reporting radiographers' (5+ years' experience) Ground truthers Ground truthing Two consultant thoracic radiologists. A third senior thoracic radiologist's opinion (\>20 years experience) will undertake arbitration.
- Primary Outcome Measures
Name Time Method Performance of AI algorithm: sensitivity During 4 weeks of reading time Evaluation of the Lunit INSIGHT CXR algorithm will be performed comparing it to the reference standard in order to determine sensitivity.
Performance of AI algorithm: specificity During 4 weeks of reading time Evaluation of the Lunit INSIGHT CXR algorithm will be performed comparing it to the reference standard in order to determine specificity.
Performance of readers with and without AI assistance: Sensitivity During 4 weeks of reading time The study will include two sessions (with and without AI overlay), with all 30 readers reviewing all 500 CXR cases each time separated by a washout period to mitigate recall bias. The cases will be randomised between the two reads and for every reader.
Performance of readers with and without AI assistance: Area under the ROC Curve (AU ROC) During 4 weeks of reading time The study will include two sessions (with and without AI overlay), with all 30 readers reviewing all 500 CXR cases each time separated by a washout period to mitigate recall bias. The cases will be randomised between the two reads and for every reader.
Performance of AI algorithm: Area under the ROC Curve (AU ROC) During 4 weeks of reading time Evaluation of the Lunit INSIGHT CXR algorithm will be performed comparing it to the reference standard. Continuous probability score from the algorithm will be utilized for the ROC analyses, while binary classification results with a predefined operating cut-off will be used for evaluation of sensitivity, specificity, positive predictive value, and negative predictive value.
Reader speed with vs without AI assistance. During 4 weeks of reading time Mean time taken to review a scan, with vs without AI assistance.
Performance of readers with and without AI assistance: Specificity During 4 weeks of reading time The study will include two sessions (with and without AI overlay), with all 30 readers reviewing all 500 CXR cases each time separated by a washout period to mitigate recall bias. The cases will be randomised between the two reads and for every reader.
- Secondary Outcome Measures
Name Time Method
Trial Locations
- Locations (1)
Oxford University Hospitals NHS Foundation Trust
🇬🇧Oxford, Oxfordshire, United Kingdom