Combined deep learning of fundus images and fluorescein angiography for retinal artery/vein classification

Authors: Sojung Go, Jooyoung Kim, Kyoung Jin Noh, Sang Jun Park, Soochahn Lee

TL;DR: A deep learning approach that jointly uses color fundus photographs and fluorescein angiography images to classify retinal vessels as arteries or veins, outperforming methods that rely on a single imaging modality.

What this paper is about

Visual overview of the proposed method with integrated feature extractor CNN for combined fundus image and FA input (FE-CNN) and hierarchical connectivity GNN (HC-GNN).

Distinguishing retinal arteries from veins is important for diagnosing vascular diseases like hypertension and diabetes. Color fundus photos show vessel structure but arteries and veins can look quite similar; fluorescein angiography (FA) captures blood-flow dynamics but is invasive. Using either modality alone leaves information on the table.

This paper is an extension of the MICCAI 2020 paper “Combining fundus images and fluorescein angiography for artery/vein classification using the hierarchical vessel graph network.”

Key idea

A visual description of the hierarchical connectivity graph neural network (HC-GNN) based on graph U-nets.

The paper proposes a combined deep learning framework that fuses features from both fundus images and fluorescein angiography to perform artery/vein (A/V) classification. By learning complementary cues from both modalities – structural appearance from fundus photos and flow-timing information from FA – the model achieves more reliable vessel labeling.

Why it mattersi

Qualitative results of sample cases from our private dataset, with top to bottom rows illustrating (1) the original image, (2) GT, and results of (3) FE-CNN only, (4) HVGN with a single graph U-net that has a two-channel output, and (5) the proposed HVGN with dual graph U-nets. Left and right columns shows the images in full and zoomed resolution, respectively.

Accurate A/V classification supports early detection of systemic vascular conditions through routine eye exams and moves toward automated retinal image analysis that leverages the full range of clinical imaging data available.