Artificial Intelligence for the Detection of Central Retinal Disease and Non-mydriatic Glaucoma in the Context of Patients With Diabetes Mellitus in Primary Care: A Prospective Study Comparing the Diagnostic Capacity of an AI Algorithm

Study Design This project followed a methodology consisting of 2 concomitant studies: In the first study, an AI algorithm was developed to detect the signs of DR in patients with diabetes. The second part of the project consisted of an observational, cross-sectional study comparing the diagnostic capacity of the algorithm with that of the family medicine physicians and with retina specialists. The reference was a blinded double reading conducted by the retina specialists (with a blinded third reading in case of disagreement in the previous 2 readings). In this way, the results obtained, both by the AI algorithm and by family medicine specialists, were compared using the gold standard (accuracy, sensitivity, specificity, area under the curve, retina specialists (with a blinded third reading in case of disagreement in the previous 2 readings). In this way, the results obtained, both by the AI algorithm and by family medicine specialists, were compared using the gold standard (accuracy, sensitivity, specificity, area under the curve, etc). The inclusion of nurses who received training in fundus readings was considered to compare their diagnostic capacity.

Study Population, Site Participation, and Recruitment. Images for the development of the algorithm were ceded by the CHS and included images from the whole Catalan population. The study took place in the primary care centers managed by the Catalan Health Institute in Central Catalonia, which includes the counties of Bages, Osona, Berguedà, and Anoia. The reference population was the population assigned to these primary care centers. This population included about 512,000 people in 2017, with an estimated prevalence of diabetes of 7.1%. The study period included 2010-2017 for the development of the algorithm with AI. Once the algorithm had been developed, the study was conducted on fundus images obtained during routine diabetic retinopathy screening over a period of about 3-4 months, between May and August 2021.

Conduct of the Study. For the development of the AI algorithm, all fundus images labeled as DR of patients from primary care centers in Catalonia between 2010 and 2017 were included. For the study, all the images of patients who underwent an eye fundus examination were included until the adequate number of patients was reached. A high percentage of the fundus images had sufficient quality; that is, a 40-degree vision of the central retina where at least a three-fourth part of the optic nerve, a well-focused macula, and well-defined veins and arteries of the upper and lower arcs can be seen. Eye fundus images that did not have adequate technical quality (dark) or that could not be evaluated due to the opacity of the media (eg, for cataracts) were excluded.

Data Collection. For the development of the AI algorithm, anonymized images with the corresponding label that classifies each image (in one of the classes with which the algorithm was trained) were required. The personnel responsible for information technology (IT) of the CHS evaluated the best strategy for the anonymization and extraction of the images from the computer systems of the CHS, as well as the identification of each image with a unique identifier. A tabulated file type CSV or TXT was used to relate each image identifier with the corresponding classification. The person responsible for IT of the CHS, together with the technical manager of OPTretina, agreed on the best way to transfer these 2 sources of information, in a secure way, from the CHS servers to the OPTretina servers (SSH File Transfer Protocol, external hard disk), depending on the volume of data to be transferred and the internal policy of the CHS. OPTretina is experienced in developing AI models for automatic fundus image classification and is a Spanish Agency of Medicines and Health Products-certified medical device manufacturer. For the study, anonymized weekly fundus readings collected by family medicine physician readers of fundus images in Central Catalonia were collected. The images were transferred to the OPTretina servers to be first analyzed by the diagnostic algorithm and then by the retina specialists who made the definitive diagnosis. The person responsible for IT of the CHS, together with the technical manager of OPTretina, agreed on the best way to transfer these data in a secure manner.