Computer Assisted Detection of Neoplasia During Colonoscopy Evaluation
- Conditions
- Colorectal Neoplasms
- Registration Number
- NCT05888623
- Lead Sponsor
- VA Puget Sound Health Care System
- Brief Summary
The purpose of this study is to evaluate the effect of artificial intelligence systems used during colonoscopy on the detection of precancerous polyps in the colon.
- Detailed Description
Background:
Colonoscopy is a key diagnostic and therapeutic procedure for the prevention of colorectal cancer (CRC) incidence and mortality. Central to colonoscopy's effectiveness is the identification and removal of colorectal neoplasia, including adenomatous polyps and sessile serrated lesions. The endoscopist's adenoma detection rate (ADR), classically defined as the proportion of screening colonoscopies in which one or more adenomas are detected, has been demonstrated to be strongly inversely associated with their patients' risk of post-colonoscopy colorectal cancer. Therefore, improving adenoma detection is a major target of quality assurance efforts.
The Veterans Health Administration's (VA) National Gastroenterology and Hepatology Program (NGHP) has embarked on a number of efforts to measure, monitor, and improve colonoscopy quality across the VA enterprise. One of these efforts is the VA Endoscopy Quality Improvement Program (VA-EQuIP) which is a collaboration between the VA Quality Enhancement Research Initiative (QUERI), the Office of Research and Development (ORD) and the NGHP. Investigators in the Measurement Science QUERI have developed processes for assessing the ADR of individual physicians and facilities through extraction of data from the Corporate Data Warehouse (CDW). Through collaboration with the VA Clinical Assessment Reporting and Tracking Program (CART), more detailed colonoscopy report information is available for 29 VA medical centers, with additional VA facilities expected to be added to the list soon. Overall, the VA ADR for colonoscopies of all indications is 47%1, well above the benchmark of 30% for men undergoing screening colonoscopy.
In 2021, the FDA approved the first artificial intelligence (AI) system for computer assisted detection (CADe) of colorectal neoplasia. These CADe devices project an image on the endoscopy monitor (i.e., a bounding box) to alert the endoscopist to the presence of a suspected polyp within the colon. Initial studies, including randomized controlled trials, have demonstrated that use of CADe systems result in a significant improvement in adenoma detection, with a reduction in the miss rate of adenomas (i.e., fewer adenomas are found on a second colonoscopy when the first colonoscopy was performed with a CADe system compared to when the first colonoscopy did not use CADe).2-4 However, more recent studies have not demonstrated a clear benefit of these devices.5,6 Moreover, most of the additional adenomas that are detected are diminutive polyps, the vast majority of which are thought to be of minimal, if any, clinical significance. When multiple adenomas are detected during colonoscopy, current guidelines recommend repeating colonoscopy sooner than would otherwise be recommended. Also, the CADe systems may have unintended consequences, such as creating alert fatigue through false alarms or negatively impacting training of gastroenterology fellows.
Objective:
The NGHP planned to formally evaluate the quantitative and qualitative impact and outcomes of use of CADe within VA medical centers following the purchase and distribution to randomly selected gastrointestinal endoscopy units.
Setting:
Veteran Health Administration (VA) Medical Centers
Study Design: Cluster Randomized Study
Intervention:
As part of ongoing quality assurance efforts, the NGHP purchased 115 Medtronic GI Genius® CADe devices in late 2022. All facilities were sorted according to their facility-level ADR (for all indications) and categorized as below 30%, 30% to \<40% or ≥40%. A random number generator was then used to sort the VA medical centers within each of these 3 ADR strata. Alternating facilities (i.e., approximately 50%) of facilities within the 2 lower ADR stratum were allocated the CADe devices. To assure equitable distribution of the remaining CADe devices across the 18 VA Integrated Service Networks (VISNs), the remaining devices were offered to randomly selected facilities (all within the ≥40% ADR stratum) such that each VISN had 2-3 facilities with CADe devices.
Upon identification of a potential site, the Gastroenterology Section Chief (or equivalent) at that randomly selected site was asked if they were interested in receiving the devices and if all endoscopists at their facility would agree to use the device during colonoscopy. The physicians were not mandated to use the devices but were asked to notify the NGHP if endoscopists were not routinely using the devices so that the devices could be reallocated to other VA facilities that would make use of the CADe devices. Upon receiving concurrence from facility endoscopy leadership, the devices were installed in all procedure rooms that are routinely used for colonoscopy at that facility. The devices were not installed on travel carts for emergency procedures. One facility did decline to receive the devices due to concerns that the devices would negatively impact efficiency in the endoscopy unit. That facility was replaced with another randomly selected VA medical center. Ultimately, 43 VA facilities received and installed the 115 CADe devices.
Survey: To assess the end-user experience, a Microsoft Forms electronic survey will be distributed to VA endoscopists who performed at least 25 colonoscopies between December 1, 2022 and May 31, 2023. Endoscopists will be invited via email with up to two follow-up reminders. Survey questions will address self-reported use of CADe for different colonoscopy indications, assessment of the benefits and negative impacts of CADe (e.g., impact on neoplasia detection or withdrawal time), and overall impression of CADe. Survey completion will be optional. Only staff gastroenterologists and surgeons will be eligible to complete the survey.
Oversight:
This quality assurance evaluation was conducted under the auspices of the VA NGHP. Evaluation of the quality of colonoscopy, including the effectiveness of CADe, was previously deemed to be quality assurance by the University of Utah and Salt Lake City VA Medical Center (IRB\_00119922); a continuing review of this IRB with amendment was approved for evaluation of AI implementation in 2022. This quality assurance evaluation is registered with ClinicalTrials.gov (NCT05888623).
Database:
The VA Corporate Data Warehouse administrative and patient care data will be used to temporally link CPT codes for colonoscopy to pathology results to assess detection of neoplasia and other study outcomes. Natural language processing (NLP) with full-text indexing searches is used to classify pathology (i.e., adenoma, sessile serrated lesion, adenocarcinoma). Validation of this method to classify adenomas from the pathology results showed 99.5% accuracy.(Gawron et al. unpublished) Separately, colonoscopy data from sites using Provation MD software (Provation, Minneapolis, MN) is exported to a national VA database for quality assurance purposes. This software includes information on colonoscopy quality, such as withdrawal time, indication, bowel preparation quality. Outcomes will be ascertained for colonoscopies performed between October 1, 2021 through June 30, 2023.
Outcomes:
Primary outcome: The primary outcome is the change in ADR from the pre- to post-deployment periods from colonoscopies performed with CADe available compared to colonoscopies performed where CADe was not available.
Secondary Outcomes:
1. Adenocarcinoma detection rate
2. Sessile serrated lesion detection rate
3. Proportion of colonoscopies where pathology specimens are obtained
4. Proportion of pathology without adenoma or adenocarcinoma
a) This serves as a surrogate for false positive lesion detection during use of CADe
5. Withdrawal time when no maneuvers are performed
1. Limited to Provation Sites
2. To assess the negative impact of CADe on procedure time or a possible Hawthorne effect.
6. Provider satisfaction with CADe for colonoscopy
7. Adenoma detection rate stratified by the endoscopist's pre-CADe ADR quintile
Additional analyses to be considered (due to data availability):
1. Bowel preparation quality
a) Limited to Provation sites
2. Number of polyps per colonoscopy
a) Limited to Provation sites
3. Size of polyps detected
a) Limited to Provation sites
4. Advanced neoplasia detection rates
a) Limited to Provation sites
5. Surveillance recommendations after colonoscopy
Independent Variables:
1. VA Facility characteristics
a) Pre-randomization adenoma detection rate
2. Patient characteristics
1. Age
2. Gender
3. Race
4. Ethnicity
5. Rurality
3. Provider characteristics
1. Specialty
2. Sex
3. Years since completion of medical training
4. Colonoscopy indication (Provation sites only, if available)
1. Screening
2. Surveillance
3. FOBT+
4. Other
5. Unknown
5. Bowel preparation quality (Provation sites only, if available)
1. Adequate vs. inadequate
2. Aronchick
3. Boston Bowel Preparation Score
Statistical Methods:
Colonoscopies performed on the day of CADe installation and the next 5 colonoscopies performed by each endoscopist after the date of installation (or matched date for control sites) will be excluded as a washout period to minimize training adaptation. For the 26 endoscopists who performed colonoscopies at multiple facilities, we will retain only procedures at the facility where they performed the largest number of colonoscopies.
Baseline data will be summarized as number (%), mean (± standard deviation) or median (interquartile range), as appropriate. For binary outcomes, we will estimate the association of CADe on study outcomes using mixed effects logistic regression (PROC GLIMMIX, SAS software, Cary, NC) with random intercepts for facility and endoscopist and random slopes (at the facility and endoscopist level) for study phase. We will also include fixed effects for group, phase, and a group by phase interaction (product term), with colonoscopy serving as the unit of analysis. The group by phase interaction is quantified as a ratio of odds ratios, where a coefficient greater than 1 for the interaction term (or 0 on the log-odds scale) reflects increased neoplasia detection in the post-deployment phase versus the pre-deployment phase in the CADe group compared to the non-CADe group. We will control for pre-randomization ADR stratum, patient demographics (e.g., sex, age, race, ethnicity and rurality), and endoscopist information (e.g., specialty, sex, years since medical degree) in all analyses.
After estimating the association of CADe on adenoma detection, we will assess whether the effect was moderated by patient and endoscopist characteristics. We will fit additional models for each potential moderator, adding a main effect for the moderator and interactions (product terms) with group, phase, and group by phase terms as fixed effects.
Withdrawal time information will only be available from the subset of VA facilities that use Provation MD software. As withdrawal time in a continuous variable, we will use a linear mixed effects regression model with the same predictors as described above. We will include only those colonoscopies without an associated pathology report and where cecal intubation was complete.
To assess whether the effect of CADe on ADR is moderated by endoscopist ADR quintile, endoscopists with at least 100 colonoscopies in the pre- and post-deployment phases each will be divided into quintiles based on pre-intervention ADR. The quintile variable will be entered into the model described above as a main effect and an interaction (product term) with the group, phase, and group by phase terms.
References:
1. Gawron AJ, Yao Y, Gupta S, et al. Simplifying Measurement of Adenoma Detection Rates for Colonoscopy. Dig Dis Sci. Sep 2021;66(9):3149-3155. doi:10.1007/s10620-020-06627-2
2. Hassan C, Spadaccini M, Iannone A, et al. Performance of artificial intelligence in colonoscopy for adenoma and polyp detection: a systematic review and meta-analysis. Gastrointest Endosc. Jan 2021;93(1):77-85 e6. doi:10.1016/j.gie.2020.06.059
3. Repici A, Badalamenti M, Maselli R, et al. Efficacy of Real-Time Computer-Aided Detection of Colorectal Neoplasia in a Randomized Trial. Gastroenterology. Aug 2020;159(2):512-520 e7. doi:10.1053/j.gastro.2020.04.062
4. Wallace MB, Sharma P, Bhandari P, et al. Impact of Artificial Intelligence on Miss Rate of Colorectal Neoplasia. Gastroenterology. Jul 2022;163(1):295-304 e5. doi:10.1053/j.gastro.2022.03.007
5. Ladabaum U, Shepard J, Weng Y, Desai M, Singer SJ, Mannalithara A. Computer-aided Detection of Polyps Does Not Improve Colonoscopist Performance in a Pragmatic Implementation Trial. Gastroenterology. Mar 2023;164(3):481-483 e6. doi:10.1053/j.gastro.2022.12.004
6. Levy I, Bruckmayer L, Klang E, Ben-Horin S, Kopylov U. Artificial Intelligence-Aided Colonoscopy Does Not Increase Adenoma Detection Rate in Routine Clinical Practice. Am J Gastroenterol. Nov 1 2022;117(11):1871-1873. doi:10.14309/ajg.0000000000001970
Recruitment & Eligibility
- Status
- COMPLETED
- Sex
- All
- Target Recruitment
- 334200
- Colonoscopy performed at a Veterans Affairs (VA) medical center
- Colonoscopy performed at VA medical centers that acquired computer-assisted detection artificial intelligence devices through non-random assignment
- Colonoscopy performed at a VA medical center where pathology results are not available
Study & Design
- Study Type
- OBSERVATIONAL
- Study Design
- Not specified
- Primary Outcome Measures
Name Time Method Adenoma Detection Rate Baseline and 6 months Change in the proportion of colonoscopies in which one or more adenomas are detected
- Secondary Outcome Measures
Name Time Method Adenocarcinoma detection rate Baseline and 6 months Change in the proportion of colonoscopies where colorectal cancer is detected
Sessile serrated lesion detection rate Baseline and 6 months Proportion of colonoscopies with one or more sessile serrated lesions detected
Proportion of pathology without adenoma or adenocarcinoma Baseline and 6 months As a surrogate for false positive lesion identification during use of CADe
Proportion of colonoscopies with pathology obtained Baseline and 6 months Proportion of colonoscopies where specimens were obtained for pathologic review
Withdrawal time without interventions Baseline and 6 months Change in the duration of colonoscope withdrawal when no intervention (e.g., polypectomy, biopsy) is performed. This outcome can only be assessed at a subset of sites due to data availability issues (i.e., Provation MD sites).
Provider satisfaction with computer assisted detection for colonoscopy Approximately 6 months after deployment Provider ratings of satisfaction with the CADe device
Trial Locations
- Locations (1)
VA Puget Sound Health Care System
🇺🇸Seattle, Washington, United States
VA Puget Sound Health Care System🇺🇸Seattle, Washington, United States