Assesment of Ethnic Bias in an Artificial Intelligence Based Orthodontic Diagnosis System

Recruiting

• Conditions: Class I Malocclusion; Class II Div 1 Malocclusion; Class II Division 2 Malocclusion; Class III Malocclusion

• Registration Number: NCT07162753
• Lead Sponsor: Bezmialem Vakif University

Brief Summary

This study explores how artificial intelligence (AI) can be used in orthodontics, which is the area of dentistry that focuses on correcting jaw and bite problems. AI is a computer technology that can learn from large amounts of data and then make predictions or decisions. It is already being tested in medicine and dentistry to help doctors and dentists diagnose conditions.

For this study, the AI system was trained using photographs and X-rays from patients in Turkey. The system learned to recognize specific orthodontic skeletal malocclusions. After the training stage, the AI was tested in two groups: one group included Turkish patients whose records were not used in training, and the other group included patients from different ethnic backgrounds who were treated at a clinic in Belgium. This design allows researchers to see if the AI works equally well for people of different backgrounds.

Only photographs and X-rays taken before orthodontic treatment are used in the study, and all data are anonymized so that no personal information is shared. The images must meet certain quality standards. For example the head must be in natural position, with no beards, scars, or previous orthodontic treatment that might affect the image. Patients who do not meet these criteria are not included.

The AI program analyzes the profile photographs, prepares them for evaluation by adjusting and standardizing the images, and then tries to decide each patient has which malocclusion. The results from Turkish patients and patients from other ethnic groups are compared to see if the system makes fair and accurate decisions for everyone.

The purpose of this study is not to test a new treatment, but to understand how well AI can recognize orthodontic problems in different populations. This information is important because AI systems are increasingly being used in healthcare, and they need to be fair and accurate for all patients, not just those from one group.

By participating, patients help researchers learn whether AI in orthodontics is reliable across diverse communities. This knowledge can guide future improvements in AI technology, ensuring that it supports orthodontists in providing safe, equal, and effective care for everyone.

Detailed Description

This study investigates how artificial intelligence (AI) can be used to diagnose skeletal malocclusions. AI is a computer technology that can learn from data, recognize patterns, and then make predictions. It is already being used in different areas of medicine and dentistry, but its accuracy and fairness may depend on the data it was trained with.

In orthodontics, skeletal problems are usually classified into four main groups: Class I (normal skeletal relationship), Class II Division 1, Class II Division 2 (upper jaw forward or lower jaw retruded or both ), and Class III (lower jaw forward in relation to the upper jaw or upper jaw retruded relation to lower jaw or both).

For this study, an AI system was trained with profile photographs and X-rays of 3,280 Turkish patients. The system learned to recognize all four types of skeletal relationships: Class I, Class II Division 1, Class II Division 2, and Class III. After the training, the AI was tested on new groups of patients whose images were not part of the training set. This included a group of Turkish patients and another group of patients from different ethnic backgrounds treated in Belgium. By comparing the AI's results in these two groups, researchers will be able to see whether the system works equally well for people of different ethnicities.

The patient data used in this study is completely anonymized. Only pre-treatment profile photographs and lateral cephalometric X-rays are included. The images must comply with quality standards such as being taken in natural head position, with a clear background and without beard, scars, or previous orthodontic treatment that might interfere with recognition. Patients whose photographs do not meet these standards are excluded.

This research is conducted as a cross-sectional diagnostic accuracy study (single-gate, multi-center). The diagnostic performance of the AI system is evaluated using accuracy, recall (sensitivity), precision, false positive rate, F1 score, and area under the curve (AUC).

The aim of this study is to evaluate whether an AI-based system used by patients for early orthodontic diagnosis provides fair and reliable results across different ethnic groups.

Study Timeline

• Study Start: 2025-01-06
• Status Verified: 2025-07
• Study First Submitted: 2025-08-26
• Study First Submitted QC: 2025-09-08
• Study First Posted: 2025-09-09
• Last Update Submitted: 2025-09-10
• Last Update Posted: 2025-09-12
• Primary Completion: 2026-06-15
• Study Completion: 2026-08-11

Recruitment & Eligibility

• Status: RECRUITING
• Sex: All
• Target Recruitment: 5000

Inclusion Criteria

Not provided

Exclusion Criteria

Not provided

Study & Design

• Study Type: OBSERVATIONAL
• Study Design: Not specified

Primary Outcome Measures

Name	Time	Method
Accuracy	At baseline (retrospective single time-point data analysis)	Accuracy (proportion of correctly classified cases) will be compared between Turkish test set, and non-Turkish cohort from different ethnic backgrounds.

Secondary Outcome Measures

Name	Time	Method
Precision (Positive Predictive Value)	At baseline (retrospective single time-point data analysis)	Percentage of cases labeled as Class III by AI that are truly Class III. Rate of AI-predicted positive cases that are truly positive, compared between Turkish group and non-Turkish ethnic group
Recall	At baseline (retrospective single time-point data analysis)	Percentage of true Class III cases correctly identified by AI
False Positive Rate	At baseline (retrospective single time-point data analysis)	Rate of negative cases incorrectly classified as positive by AI, compared across cohorts.
F1 score	At baseline (retrospective single time-point data analysis)	Harmonic mean of precision and recall (range 0-1). Higher values indicate better overall performance, while lower values indicate poor balance between sensitivity and precision.

Trial Locations

Locations (1)

Bezmialem Vakıf University; Istanbul, Fatih, Turkey (Türkiye)