Graph-Aware Multimodal Deep Learning for Classification of Diabetic Retinopathy Images

Diabetic Retinopathy (DR) is one of the leading causes of blindness worldwide, where early detection is critical to preventing irreversible vision loss. Traditional diagnostic methods primarily rely on single-modality data, such as retinal images, or the analysis of image features, which may limit diagnostic accuracy. To overcome these limitations, we propose Diabetic Retinopathy Diagnosis, namely “DRdiag”, a novel model proposed for the early diagnosis of DR. DRdiag integrates multiple modalities: the Fundus images and the duration of disease evolution which is an important factor in the diagnosis of DR as the duration of the disease directly influences the onset and progression of retinal lesions, leveraging two distinct models: a Convolutional Neural Network (CNN) based on DenseNet121 for image feature extraction and a Graph Neural Network (GNN) for capturing complex relationships between patient features. By combining the strengths of these advanced models, DRDiag aims to provide a more comprehensive and accurate diagnostic system, ultimately supporting timely intervention and improved patient outcomes. The experiments are conducted using two datasets, APTOS2019 and Messidor-2, which provide diverse retinal images for robust evaluation. The proposed model achieves high performance on diabetic retinopathy classification. On Messidor-2, it attains 0.976 accuracy, 0.957 kappa, 0.961 F1-score, 0.958 recall, 0.965 precision, and 0.990 specificity. On APTOS2019, it achieves 0.980 accuracy, 0.960 kappa, 0.963 F1-score, 0.959 recall, 0.968 precision, and 0.995 specificity. These results confirm its robustness and reliability.