Vision-Language Neural Graph Featurization for Extracting Retinal Lesions

Abstract

Retinopathy comprises a group of retinal disorders that can lead to severe visual impairments or blindness. The heterogeneous morphology of lesions poses a significant challenge in developing robust diagnostic systems. Supervised approaches rely on large labeled datasets and often struggle with generalization. To address these limitations, we propose an unsupervised vision-language neural graph featurization method. This method first segments fundus images into a set of superpixels via Simple Linear Iterative Clustering (SLIC). The superpixels are then decomposed into an undirected graph where each superpixel serves as a node, and spatially adjacent nodes are connected by edges. A Hamiltonian path systematically traverses the graph and iteratively updates and propagates node and edge latent space embeddings throughout the graph until convergence is achieved. Then, a normalized cut separates the converged embeddings into two clusters within a latent space that represent the lesion and healthy superpixels of the input scans. The lesion superpixels are further classified into lesion categories using a prompt-based zero-shot vision-language model. The proposed method is rigorously tested on four public datasets, dubbed ODIR, FIVES, BIOMISA, and IDRiD, achieving F1-scores of 0.89, 0.92, 0.93, and 0.92, respectively, with significant performance gains over state-of-the-art methods.