Retrospective Study of Carebot AI CXR Performance in Preclinical Practice

Completed

• Conditions: Hilar Calcification; Artificial Intelligence; Cardiomegaly; Pulmonary Edema; Lung Diseases; Pneumothorax; Lung Cancer; Atelectasis; Pleural Effusion; Consolidation
• Interventions: Device: Carebot AI CXR

• Registration Number: NCT05594485
• Lead Sponsor: Carebot s.r.o.

Brief Summary

The purpose of this study is to describe the design, methodology and evaluation of the preclinical test of Carebot AI CXR software, and to provide evidence that the investigated medical device meets user requirements in accordance with its intended use. Carebot AI CXR is defined as a recommendation system (classification "prediction") based on computer-aided detection. The software can be used in a preclinical deployment at a selected site before interpretation (prioritization, display of all results and heatmaps) or after interpretation (verification of findings) of CXR images, and in accordance with the manufacturer's recommendations. Given this, a retrospective study is performed to test the clinical effectiveness on existing CXRs.

Detailed Description

The performance of the trained and internally validated Carebot AI CXR software is tested on a set of 127 CXR images from target population. This is compared to common clinical practice, i.e., image assessment by a radiologist in a hospital. Patients may have a variety of findings; at this stage of the evaluation, an abnormal finding is considered to be an abnormality in any of the defined classes. False negative images incorrectly predicted by Carebot AI CXR software result in a clinical impact determination.

To collect the CXR data for retrospective study, investigators addressed a municipal hospital in the Czech Republic that provides healthcare services to up to 130,000 residents of a medium-sized city (approximately 70,000 inhabitants) and the surrounding area. 127 anonymized CXR images were collected between August 15 and 17, 2022, and subsequently submitted to five independent radiologists of varying experience for annotation. The selected radiologists were asked to assess whether the CXR image shows any of the 12 pre-selected abnormalities. Pediatric CXR images (under 18 years of age), scans with technical problems (poor image quality, rotation), and images in lateral projection were excluded from the dataset.

Study Timeline

• Study Start: 2022-08-15
• Primary Completion: 2022-08-17
• Study Completion: 2022-10-20
• Study First Submitted: 2022-10-21
• Study First Submitted QC: 2022-10-21
• Study First Posted: 2022-10-26
• Status Verified: 2023-07
• Last Update Submitted: 2023-07-12
• Last Update Posted: 2023-07-13

Recruitment & Eligibility

• Status: COMPLETED
• Sex: All
• Target Recruitment: 127

Inclusion Criteria

• Hospital patients who were referred for chest radiography between August 15 and 17, 2022.

Exclusion Criteria

• Pediatric CXR images (under 18 years of age)
• Scans with technical problems (poor image quality, rotation)
• Images in lateral projection

Study & Design

• Study Type: OBSERVATIONAL
• Study Design: Not specified

Arm && Interventions

Group	Intervention	Description
Retrospective collection of DICOM patient files for the period 15-17 August	Carebot AI CXR	To collect the CXR data for retrospective study, we addressed a municipal hospital in the Czech Republic that provides healthcare services to up to 130,000 residents of a medium-sized city (approximately 70,000 inhabitants) and the surrounding area. 127 anonymized CXR images were collected between August 15 and 17, 2022, and subsequently submitted to five independent radiologists of varying experience for annotation. The selected radiologists were asked to assess whether the CXR image shows any of the 12 abnormalities mentioned above. Pediatric CXR images (under 18 years of age), scans with technical problems (poor image quality, rotation), and images in lateral projection were excluded from the dataset.

Primary Outcome Measures

Name	Time	Method
Primary objective	20-10-2022	Comparison of the accuracy of radiologist and Carebot AI CXR image assessment.

Secondary Outcome Measures

Name	Time	Method
Secondary objective	20-10-2022	Comparison of the accuracy of radiologis with different experience vs. Carebot AI CXR. Weakness assessment of Carebot AI CXR.

Trial Locations

Locations (1)

Nemocnice Havířov, p. o.; 🇨🇿 Havířov, Czechia