Clinical Validity of an AI-Based Biological Signal Analysis Software Vital-PICASO for Predicting the Risk of Hypoxia Within 24 Hrs Using Vital Sign (SBP, DBP, HR, RR, BT, SpO₂) From General Ward Inpatients; A Retrospective, Single-Center, Single-Arm, Superiority-confirmatory Clinical Trial
Trial Snapshot
- Phase
- Not Applicable
- Status
- Completed
- Sponsor
- Enrollment
- 2,250
- Locations
- 1
- Primary Endpoint
- Area Under the Receiver Operating Characteristic Curve for SpO2-Based Hypoxia Prediction
Study Overview
Brief Summary
The purpose of this retrospective study is to evaluate the clinical performance of Vital-PICASO, an artificial intelligence-based biological signal analysis software designed to predict the risk of hypoxia within 24 hours using vital-sign data from adult general ward inpatients.
Electronic medical record data from patients aged 19 years or older who were admitted to a general ward at Seoul National University Hospital will be retrospectively reviewed. Hypoxia will be evaluated using two separate reference-standard criteria: oxygen saturation below 94% and administration of oxygen at 4 L/min or more.
Eligible hypoxia-positive and hypoxia-negative datasets will be randomly selected. The selected vital-sign datasets will be analyzed using Vital-PICASO while the device operator is blinded to the reference-standard classification. The software-generated hypoxia risk scores will be compared with the reference-standard classifications to evaluate predictive performance. Because the study uses previously collected medical records, there is no direct participant contact and the software results will not affect patient care.
Detailed Description
This study is a retrospective, single-center, superiority-confirmatory clinical performance study conducted at Seoul National University Hospital.
Data Source and Study Population
Electronic medical records will be reviewed for male and female patients aged 19 years or older who were admitted to a general ward for at least 12 hours between January 1, 2023, and May 31, 2025. Clinical data previously used in the development of Vital-PICASO will be excluded.
The collected variables will include systolic blood pressure, diastolic blood pressure, heart rate, respiratory rate, body temperature, oxygen saturation, and the corresponding measurement times. Additional variables will include the occurrence, time, and location of hypoxia; oxygen administration status, flow rate, delivery method, and start time; month and year of birth; sex; height and weight when available; do-not-resuscitate status; chronic obstructive pulmonary disease status; pregnancy status; date of admission to the general ward; clinical department; and the vital-sign data collection period.
Reference-Standard Groups
Study Design
- Study Type
- Observational
- Observational Model
- Case Control
- Time Perspective
- Retrospective
Eligibility Criteria
- Ages
- 19 Years to — (Adult, Older Adult)
- Sex
- All
- Accepts Healthy Volunteers
- No
Inclusion Criteria
- •General Criteria
- •Male or female patient aged 19 years or older at the time the clinical data were recorded
- •Electronic medical record data available between January 1, 2023, and May 31, 2025
- •Admission to a general ward for at least 12 hours
- •Clinical data not previously used in the development of Vital-PICASO
- •SpO2-Based Hypoxia-Positive Group
- •Hypoxia occurred in a general ward
- •Hypoxia was defined as oxygen saturation below 94%
- •Vital-sign data were available for up to 72 hours before the hypoxia event
- •If hypoxia occurred within 24 hours after admission or transfer to the general ward, all available vital-sign data recorded before the event were included
- •If multiple eligible hypoxia events occurred during the data collection period, only the first eligible event was included for this criterion
- •SpO2-Based Hypoxia-Negative Group
- •No hypoxia occurred during the selected general ward admission
- •Data were randomly selected from the same general wards as the corresponding hypoxia-positive group
- •Vital-sign data were available from a time interval comparable with the hypoxia-event reference time or for up to 72 hours after admission to the general ward
- •Only one eligible data interval was included per patient
- •Oxygen-Treatment-Based Hypoxia-Positive Group
- •Hypoxia occurred in a general ward
- •Hypoxia was defined as oxygen administration at a flow rate of 4 L/min or more
- •Vital-sign data were available for up to 72 hours before the initiation of oxygen administration
- •If oxygen administration began within 24 hours after admission or transfer to the general ward, all available vital-sign data recorded before the event were included
- •If multiple eligible events occurred during the data collection period, only the first eligible event was included for this criterion
- •Oxygen-Treatment-Based Hypoxia-Negative Group
- •No hypoxia occurred during the selected general ward admission
- •Data were randomly selected from the same general wards as the corresponding hypoxia-positive group
- •Vital-sign data were available from a time interval comparable with the hypoxia-event reference time or for up to 72 hours after admission to the general ward
- •Only one eligible data interval was included per patient
Exclusion Criteria
- •Hypoxia occurred outside a general ward
- •The patient had chronic obstructive pulmonary disease
- •A do-not-resuscitate order was in effect at the time the clinical data were recorded
- •The patient was pregnant or within 3 months after delivery at the time the clinical data were recorded
- •The data were considered inappropriate for the study by the principal investigator
Outcomes
Primary Outcomes
Area Under the Receiver Operating Characteristic Curve for SpO2-Based Hypoxia Prediction
Time Frame: Within 24 hours before the SpO2-based hypoxia event or during the corresponding reference period for negative cases
The area under the receiver operating characteristic curve (AUROC) is calculated by comparing the hypoxia risk scores generated by Vital-PICASO with the reference-standard classification for the SpO2-based analysis. Hypoxia-positive cases are defined as cases in which oxygen saturation below 94% occurred in the general ward. Hypoxia-negative cases are selected from patients who did not experience hypoxia during the corresponding general ward admission period. AUROC ranges from 0 to 1, with a higher value indicating better discrimination between hypoxia-positive and hypoxia-negative cases. The prespecified success criterion is met if the lower bound of the 95% confidence interval for the difference between the Vital-PICASO AUROC and the comparator AUROC of 0.7891 is greater than 0.
Area Under the Receiver Operating Characteristic Curve for Oxygen-Treatment-Based Hypoxia Prediction
Time Frame: Within 24 hours before oxygen administration at 4 L/min or more or during the corresponding reference period for negative cases
The area under the receiver operating characteristic curve (AUROC) is calculated by comparing the hypoxia risk scores generated by Vital-PICASO with the reference-standard classification for the oxygen-treatment-based analysis. Hypoxia-positive cases are defined as cases in which oxygen was administered at a flow rate of 4 L/min or more in the general ward. Hypoxia-negative cases are selected from patients who did not experience hypoxia during the corresponding general ward admission period. AUROC ranges from 0 to 1, with a higher value indicating better discrimination between hypoxia-positive and hypoxia-negative cases. The prespecified success criterion is met if the lower bound of the 95% confidence interval for the difference between the Vital-PICASO AUROC and the comparator AUROC of 0.7891 is greater than 0.
Secondary Outcomes
- Area Under the Precision-Recall Curve for SpO2-Based Hypoxia Prediction(Within 24 hours before the SpO2-based hypoxia event or during the corresponding reference period for negative cases)
- Area Under the Precision-Recall Curve for Oxygen-Treatment-Based Hypoxia Prediction(Within 24 hours before oxygen administration at 4 L/min or more or during the corresponding reference period for negative cases)
- Precision, Recall, and F1 Score Across Risk-Score Thresholds(Within the 24-hour prediction window before the hypoxia event or corresponding reference period)
- Predictive Performance by Prediction-Time Interval(At prespecified time intervals within 24 hours before the hypoxia event or corresponding reference time point)
- Predictive Performance by Cumulative Prediction Window(Over cumulative prediction windows of up to 24 hours before the hypoxia event or corresponding reference time point)
- Sensitivity, Specificity, Positive Predictive Value, and Negative Predictive Value(Within the 24-hour prediction window before the hypoxia event or corresponding reference period)