Breast Ultrasound Image Reviewed With Assistance of Deep Learning Algorithms

Not Applicable

• Conditions: Breast Mass; Breast Lesions; Breast Cancer
• Interventions: Device: Ultrasound Image review with CADe; Device: Ultrasound Image review with CADx; Device: Ultrasound Image manual review; Procedure: Biopsy

• Registration Number: NCT03706534
• Lead Sponsor: Samsung Medison

Brief Summary

This study evaluates a second review of ultrasound images of breast lesions using an interactive "deep learning" (or artificial intelligence) program developed by Samsung Medical Imaging, to see if this artificial intelligence will help the Radiologist make more accurate diagnoses.

Detailed Description

Using ultrasound images prospectively acquired, the purpose of this study entails a second review of ultrasound images with suspicious breast lesions using an interactive "deep learning" (or artificial intelligence) program developed by SamsungMedison Co.,Ltd.

The images will be reviewed by the radiologists twice: first without, and then with assistance of artificial intelligence program by SamsungMedison Co., Ltd.

BIRADS system will be used in this study.

The objectives of the study are twofold: to quantify the statistical equivalence of radiologists' opinion and AI's output (CADe), and to check BIRADS score-based diagnostic accuracy (CADx) that is gained by the Radiologists' use of this interactive tool

Study Timeline

• Study Start: 2018-09-20
• Study First Submitted: 2018-10-11
• Study First Submitted QC: 2018-10-11
• Study First Posted: 2018-10-16
• Status Verified: 2019-10
• Last Update Submitted: 2019-10-27
• Last Update Posted: 2019-10-29
• Primary Completion: 2019-11-30
• Study Completion: 2020-01-31

Recruitment & Eligibility

• Status: UNKNOWN
• Sex: All
• Target Recruitment: 300

Inclusion Criteria

• Adult females or males recommended for ultrasound-guided breast lesion biopsy or ultrasound follow-up with at least one suspicious lesion
  • Age > 18 years
  • Able to provide informed consent

Exclusion Criteria

• Unable to read and understand English
  • Unable or unwilling to provide informed consent
  • A patient with current or previous diagnosis of breast cancer in the same quadrant
  • Unable or unwilling to undergo study procedures

3. Subject Characteristics

   1. Number of Subjects: 300 subjects from 300 separate breast lesions can be acquired. If a subject has more than 1 suspicious lesion, each may be chosen by the radiologist attending as suitable for "second review".
   2. Gender and Age of Subjects: Adult females or males aged 18 years or older who meet all of the inclusion criteria and none of the exclusion criteria will be considered for enrollment. Minors are excluded as breast cancer is very rare in this age group.
   3. Racial and Ethnic Origin: There are no enrollment exclusions based on economic status, race, or ethnicity. Based on local and United States census data, the expected ethnic distribution will be approximately 26 Hispanic (approx. 16%) and 134 non-Hispanic people. Furthermore, the expected racial distribution is expected to be approximately 126 White (approx. 79% of the whole study), 21 Black or African America (13%), 8 Asian (5%), and 5 of other categories (3%).
   4. Vulnerable Subjects: It is unlikely that any UR students or employees will be enrolled unless their primary physician refers them to UR Medicine Breast Imaging at Red Creek for breast ultrasound and a suspicious lesion is found. We do not expect any of these referrals to be from staffs who work directly with the PIs.

Study & Design

• Study Type: INTERVENTIONAL
• Study Design: CROSSOVER

Arm && Interventions

Group	Intervention	Description
Review by S-Detect for Breast	Ultrasound Image review with CADe	The same images will be separately processed by the artificial intelligence system (S-Detect for Breast) by Samsung. The two results, one by the radiologists and the other by artificial intelligence system, will be compared to statistically quantify equivalence (CADe).
Manual review	Biopsy	The images will be reviewed by the radiologists using BIRADS scheme without any assistance of artificial assistance. This review will be done off-line using a separate program in entirely manual mode. During this review, BIRADS descriptor choices by each radiologist and the time it takes for the radiologist to make such decision will be stored. Radiologists also make assessment decision without any intervention from artificial intelligence. 10 radiologists review manually.
Review with assistance of S-Detect for Breast	Ultrasound Image review with CADx	Second, the images will be reviewed by the radiologists with the help of artificial intelligence system, which is an interactive tool automatically providing recommendations on BIRADS descriptor choices that can be modified by the radiologists. The radiologists, after selecting all the descriptors of BIRADS, will decide the assessment categories. These decisions will be compared with the ground truths generated from the biopsy results or a 24-month follow-up (CADx).
Review by S-Detect for Breast	Ultrasound Image manual review	The same images will be separately processed by the artificial intelligence system (S-Detect for Breast) by Samsung. The two results, one by the radiologists and the other by artificial intelligence system, will be compared to statistically quantify equivalence (CADe).
Review with assistance of S-Detect for Breast	Ultrasound Image manual review	Second, the images will be reviewed by the radiologists with the help of artificial intelligence system, which is an interactive tool automatically providing recommendations on BIRADS descriptor choices that can be modified by the radiologists. The radiologists, after selecting all the descriptors of BIRADS, will decide the assessment categories. These decisions will be compared with the ground truths generated from the biopsy results or a 24-month follow-up (CADx).
Manual review	Ultrasound Image review with CADe	The images will be reviewed by the radiologists using BIRADS scheme without any assistance of artificial assistance. This review will be done off-line using a separate program in entirely manual mode. During this review, BIRADS descriptor choices by each radiologist and the time it takes for the radiologist to make such decision will be stored. Radiologists also make assessment decision without any intervention from artificial intelligence. 10 radiologists review manually.
Manual review	Ultrasound Image manual review	The images will be reviewed by the radiologists using BIRADS scheme without any assistance of artificial assistance. This review will be done off-line using a separate program in entirely manual mode. During this review, BIRADS descriptor choices by each radiologist and the time it takes for the radiologist to make such decision will be stored. Radiologists also make assessment decision without any intervention from artificial intelligence. 10 radiologists review manually.
Manual review	Ultrasound Image review with CADx	The images will be reviewed by the radiologists using BIRADS scheme without any assistance of artificial assistance. This review will be done off-line using a separate program in entirely manual mode. During this review, BIRADS descriptor choices by each radiologist and the time it takes for the radiologist to make such decision will be stored. Radiologists also make assessment decision without any intervention from artificial intelligence. 10 radiologists review manually.
Review with assistance of S-Detect for Breast	Biopsy	Second, the images will be reviewed by the radiologists with the help of artificial intelligence system, which is an interactive tool automatically providing recommendations on BIRADS descriptor choices that can be modified by the radiologists. The radiologists, after selecting all the descriptors of BIRADS, will decide the assessment categories. These decisions will be compared with the ground truths generated from the biopsy results or a 24-month follow-up (CADx).

Primary Outcome Measures

Name	Time	Method
Concordance rate	2 days	Breast Imaging Reporting and Data System descriptors suggested by S-Detect for Breast are in good agreement with those selected by experts. In other words, the Breast Imaging Reporting and Data System Lexicon values generated by S-Detect for Breast are not statistically different from the consensus of experts.; Breast Imaging Reporting and Data System Assessment Category Score: The user makes the final decision on the Assessment Category Score. Using this Score, S-Detect displays the assessment description.; Category 0: Incomplete - Need Additional Imaging Evaluation Category 1: Negative Category 2: Benign Category 3: Probably Benign Category 4a: Low suspicion for malignancy Category 4b: Moderate suspicion for malignancy Category 4c: High suspicion for Malignancy Category 5: Highly Suggestive of Malignancy Category 6: Known Biopsy-Proven Malignancy

Secondary Outcome Measures

Name	Time	Method
Accuracy	7 day	Comparing to the Breast Biopsy results, The accuracy of Breast Imaging results by radiologists with CADx will be evaluated.
Sensitivity	7 day	Comparing to the Breast Biopsy results, The sensitivity of Breast Imaging results by radiologists with CADx will be evaluated.
Area Under Curve	7 day	Comparing to the Breast Biopsy results, Area Under Curve (ROC analysis) of Breast Imaging results by radiologists with CADx will be evaluated.
Consensus	2 day	Evaluate the consensus between manually reading of Breast Imaging without assistance and Automatically detection results(Breast Imaging Reporting and Data System Lexicons). Average of consensus is evaluated in both of Expert group and non-expert group.
Reporting time	2 day	Measure reporting time of Breast Imaging Reporting and Data System Lexicon value in Breast imaging by radiologists without S-Detect for Breast and also measured report time by radiologists with S-Detect for Breast.
Specificity	7 day	Comparing to the Breast Biopsy results, The specificity of Breast Imaging results by radiologists with CADx will be evaluated.

Trial Locations

Locations (1)

University of Rochester; 🇺🇸 Rochester, New York, United States