AI-based System for Assessing Suspected Viral Pneumonia Related Lung Changes
- Conditions
- Pneumonia, ViralCOVID-19 Pneumonia
- Interventions
- Diagnostic Test: Medical software (AI-based system)
- Registration Number
- NCT06501599
- Lead Sponsor
- Research and Practical Clinical Center for Diagnostics and Telemedicine Technologies of the Moscow Health Care Department
- Brief Summary
The AI-based system designed to process chest computed tomography (CT) aims to 1) detect the presence of pathologic patterns associated with interstitial changes in pneumonia; 2) highlight areas on the images with the probable presence of pathologies; 3) provide the physician with the results of image processing, including quantitative indicators of suspected viral pneumonia related lung changes according to visual pulmonary lesion grading system (CT0-4).
The retrospective study aims to demonstrate the clinical validation of the AI-based system. Clinical validation measures (sensitivity, specificity, accuracy, and area under the ROC curve) will be determined to provide evidence about the clinical efficacy of the AI-based system.
The hypothesis is that the measures of clinical validation of the AI-based system differ by no more than 8% from those declared by the manufacturer.
- Detailed Description
The AI-based system designed to process chest CT aims to 1) detect the presence of pathologic patterns associated with interstitial changes in pneumonia; 2) highlight areas on the images with the probable presence of pathologies; 3) provide the physician with the results of image processing, including quantitative indicators of suspected viral pneumonia related lung changes according to visual pulmonary lesion grading system (CT0-4).
This retrospective clinical study will provide the clinical validation of the AI-based system to analyze chest CT images and identify pathological patterns associated with interstitial changes in pneumonia. Clinical validation measures (sensitivity, specificity, accuracy, and area under the ROC curve) will be determined and compared with values declared by the manufacturer to provide evidence about the clinical efficacy of the AI-based system.
The first stage of clinical validation is the collection of a verified labeled dataset. For this purpose, the dataset is collected, labeled, and verified by a research group. The verified dataset should include chest CT images without infiltrative and interstitial lung changes characteristic of viral pneumonia, including COVID-19-associated (CT-0) and chest CT images of all degrees of lung involvement CT-1 (≤25%), CT-2 (25-50%), CT-3 (50-75%), CT-4 (≥75%) \[1\]. Forming the verified dataset will allow reliable conclusions to be drawn upon completion of the clinical validation. The verified dataset must include a sufficient volume of chest CT images. The verified dataset must be de-identified to ensure the safety of patient personal data.
The second stage of the clinical validation is assessing AI-based system performance by experts. For that purpose, the AI software is analyzed to identify radiological signs of viral pneumonia. Then an examination is made of the correctness of the quantitative assessment of lung damage associated with interstitial changes in pneumonia. The evaluation of both the ability to correctly identify signs of lung damage and to quantify the identified changes is carried out on the same verified dataset.
The third stage of clinical validation is the calculation of clinical efficacy metrics (accuracy, sensitivity, specificity, area under the ROC-curve (AUROC) of the AI-based system by testing it on a verified data set. Testing of the hypothesis to verify the main diagnostic characteristics (sensitivity and specificity) declared by the manufacturer is planned by constructing a two-sided 95% confidence interval (CI), which should not differ by more than 8% from the declared values of 95% and 97%, respectively. Those. the lower limit of the 95% CI for sensitivity should not cross the 87% threshold, and the lower limit of the 95% CI for specificity should not cross the 89% threshold.
All stages of the clinical trial must be under the control of the Principal Investigator.
Randomization of images is not provided in this clinical study, because All CT images will be assessed by the research group and AI software. Also, this design does not involve blinding or masking of the research team. The evaluation of CT images by experts and the software is carried out independently, i.e. the results of each party's assessment are not known to the other party in advance.
Recruitment & Eligibility
- Status
- RECRUITING
- Sex
- All
- Target Recruitment
- 563
-
General
- Patients over 18 years old;
- Patients who underwent CT without contrast enhancement;
- Patients who underwent a CT scan according to a standardized scanning protocol: 120 kilovolts, slice thickness max. 2 mm, rigid "lung" filter (kernel) reconstruction;
- Patients whose studies should be of acceptable quality, performed with breath-holding, without technical artifacts, and respiratory and motor artifacts;
- Patients whose studies must contain DICOM tags responsible for the orientation and position of the patient in the images during the study, as well as DICOM tags responsible for the size of the scans and image parameters;
- Patients in whom the localization of changes is predominantly bilateral, in the basal and subpleural parts of the lungs, may be located peribronchial;
-
For group Normal
a. Patients who do not contain COVID-19-related CT patterns;
-
For groups Mild, Moderate, Severe, and Critical
- Patients who contain COVID-19-related CT pattern: ground glass opacities (mild, moderate, and higher intensity);
- Patients who contain COVID-19-related CT pattern: pulmonary consolidation;
- Patients who contain COVID-19-related CT pattern: cobblestone infiltration of the lung parenchyma;
- Patients who contain COVID-19-related CT pattern: hydrothorax;
- Patients who contain a combination of one or more patterns.
- Patients whose studies contain images with unreported CT patterns;
- Patients whose examinations do not conform to DICOM format;
- Patients whose examinations do not contain imaging of the lung region
- Patients whose examinations contain technical artifacts caused by malfunctions or features of CT scanners;
- Patients whose examinations contain improper patient positioning;
- Patients whose examinations contain studies with deleted DICOM tags responsible for scan size and image parameters;
- Patients whose examinations contain metal artifacts on the patient's body and clothing;
- Patients whose examinations contain the presence of other pathologic changes of lungs in patients - neoplastic, tuberculosis process, bacterial pneumonia, etc.;
- Patients under 18 years old.
Study & Design
- Study Type
- OBSERVATIONAL
- Study Design
- Not specified
- Arm && Interventions
Group Intervention Description Normal Medical software (AI-based system) CT-0. Not consistent with pneumonia (including COVID-19). \[1,2\] Enrollment number: 95 Moderate Medical software (AI-based system) CT-2. Ground glass opacities. Pulmonary parenchymal involvement 25-50% \[1,2\]. Enrollment number: 117 Severe Medical software (AI-based system) CT-3. Ground glass opacities. Pulmonary consolidation. Pulmonary parenchymal involvement 50-75%. Lung involvement increased in 24-48 hours by 50% in respiratory impairment per follow-up studies \[1,2\]. Enrollment number: 117 Critical Medical software (AI-based system) CT-4. Diffuse ground glass opacities with consolidations and reticular changes. Hydrothorax (bilateral, more on the left). Pulmonary parenchymal involvement \>=75% \[1,2\]. Enrollment number: 117 Mild Medical software (AI-based system) СT-1. Ground glass opacities. Pulmonary parenchymal involvement =\<25% OR absence CT signs in typical clinical manifestations and relevant epidemiological history \[1,2\]. Enrollment number: 117
- Primary Outcome Measures
Name Time Method Sensitivity Upon completion, up to 1 year Effectiveness of the AI-based system to correctly identifies patients with the suspected viral pneumonia related lung changes
Accuracy Upon completion, up to 1 year The ability of an AI-based system to produce the correct result relative to the total number of trials
AUC ROC Upon completion, up to 1 year The area under curve (AUC) of Receiver Operating Characteristic (ROC) curves of AI-based system in prediction of suspected viral pneumonia related lung changes
Specificity Upon completion, up to 1 year Effectiveness of the AI-based system to correctly identifies across a range of available measurements patients that do not have the suspected viral pneumonia related lung changes
- Secondary Outcome Measures
Name Time Method Approximate volume of affected lung tissue Time Frame: Upon completion, up to 1 year Approximate volume of affected lung tissue - quantitative characteristics of lung damage volume in percent (%): separately for left lung, right lung and total percentage of damage
Trial Locations
- Locations (1)
Research and Practical Clinical Center for Diagnostics and Telemedicine Technologies of the Moscow Health Care Department
🇷🇺Moscow, Russian Federation