Validation of the Fracture Classification Accuracy (Ground Truth) of Anteroposterior X-ray of the Proximal Femur According to the Arbeitsgemeinschaft für Osteosynthesefragen/Orthopedic Trauma Association Classification Done by a Single Center: A Pilot Validation Study

The AO@AI Turin project is a collaborative project with a Turin group who has collected 2,932 anteroposterior (AP) pelvic radiographs, of which 1,811 are fracture images, and 1,121 are non-fracture images. The Turin group has developed an artificial intelligence (AI) algorithm for fracture classification using these images. These anonymized images (with all metadata or personal identifiers removed) have been uploaded to a cloud-based image database (DB) hosted and managed by the AO Foundation.

The Turin group has established the "ground truth" using the methods of "consensus by experts". Two radiologists from their medical team have reviewed and classified the fracture status (fracture vs non-fracture, and, if fracture, the AO/Orthopedic Trauma Association [OTA] classification).

The next step's goal is the ground truth validation plan to test the accuracy of the Turin annotation of fracture classification of the already uploaded AP pelvic images. This is to ensure that the image DB offers accurate quality annotations to allow AI development.

For the pilot phase, a random subset of the Turin images (300 of images) will be drawn from the image DB. These images will be reviewed by an external group of AO expert surgeons who will annotate the images per their fracture status, i.e., fracture vs non-fracture, and, if fracture, the AO/OTA classification.

The group of AO expert surgeons consists of four surgeons who will independently review the 300 images and a fifth surgeon who serves as an adjudicator if necessary. The expert surgeons will be given access to the 300 images via the cloud-based image DB and annotate the images. The expert surgeons will be blinded to the Turin annotations. The expert surgeons' annotations will be entered into a DB built for the purpose for the pilot study.