Validation of an Artificial Intelligence-based Algorithm for Skeletal Age Assessment

Not Applicable

Completed

Brief Summary: The purpose of this study is to understand the effects of using an Artificial Intelligence algorithm for skeletal age estimation as a computer-aided diagnosis (CADx) system. In this prospective real-time study, the investigators will send de-identified hand radiographs to the Artificial Intelligence algorithm and surface the output of this algorithm to the radiologist, who will incorporate this information with their normal workflows to make an estimation of the bone age. All radiologists involved in the study will be trained to recognize the surfaced prediction to be the output of the Artificial Intelligence algorithm. The radiologists' diagnosis will be final and considered independent to the output of the algorithm.

Detailed Description: The investigators are targeting to study the effect of their Artificial Intelligence algorithm on the radiologists' estimation of skeletal age. Currently, radiologists make the estimation using only the radiographic images and health records. As part of this study, the radiologists will estimate skeletal age from radiographic images, health records, and the output of the CADx algorithm. The investigators wish to understand how radiologists using the Artificial Intelligence algorithm compare to radiologists who do not for the specific task of estimating skeletal age.

This study is organized as a multi-institutional randomized control trial with two arms - experiment (receiving the Artificial Intelligence algorithm's output) and control (no intervention). Both of these arms will be compared to a clinical reference standard ("gold standard") composed of a panel of radiologists. The metric of comparison will be Mean Absolute Distance (MAD). The investigators plan to use statistical tests such as the t-test to determine any statistically-significant difference in skeletal age estimation between the two groups.

The investigators have recruited and analyzed data from a sample size of 1600 exams. Patients getting these exams will not undergo any research procedures that deviate from the current standard practices.

Recruitment & Eligibility

Inclusion Criteria

Not provided

Exclusion Criteria

Not provided

Arm && Interventions

Group	Intervention	Description
Experiment (With-AI)	BoneAgeModel	This is the experiment arm where the intervention, "BoneAgeModel", is provided. The participating radiologists in this arm will receive the output of the Artificial Intelligence algorithm. They will be asked to incorporate this new information with their normal workflows to make a diagnosis. The radiologists' diagnosis will be considered final.

Primary Outcome Measures

Name	Time	Method
Paired Difference of Skeletal Age Estimate	Up to 10 minutes to acquire the scan; up to 2 days to complete diagnosis review	Mean absolute difference between dictated final impressions (baseline measure by Radiologist) and the consensus determination of a panel of radiologists following review.

Secondary Outcome Measures

Name	Time	Method
Time for Diagnosis	Up to approximately 4 minutes	Amount of time taken by radiologists when using the BoneAgeModel as compared to when they are not.

Locations (6): Stanford University
🇺🇸
Stanford, California, United States
Boston Children's Hospital
🇺🇸
Boston, Massachusetts, United States
New York University
🇺🇸
New York, New York, United States
Cincinnati Children's Hospital Medical Center
🇺🇸
Cincinnati, Ohio, United States
Children's Hospital of Philadelphia
🇺🇸
Philadelphia, Pennsylvania, United States
Yale New Haven Hospital
🇺🇸
New Haven, Connecticut, United States