The FIREFLY Dataset
(Fundus Images REgistered with FLuorescein angiographY)
The FIREFLY dataset
The Fundus Images REgistered with FLuorescein angiographY dataset is constructed by combining fluorescein angiography (FA) images together with fundus images to construct highly detailed and accurate ground truth of blood vessel segmentation masks as well as artery/vein (A/V) masks.
The process comprises two subsequent steps of constructing the vessel segmentation mask and constructing the A/V mask.
-
For vessel segmentation, we apply registration techniques are applied to aggregate the FA frames and to align the aggregated FA with the fundus image, as well as CNN-based segmentation, resulting in a fine-scale detailed segmentation, based on FA, aligned to the fundus image [1,2].
-
For A/V classification, we apply GNN-based classification on pixel-wise CNN features of vessel pixels, for the artery tree and vein tree, respectively [3,4].
-
For both steps, the resulting vessel segmentation mask and A/V masks undergo a manual editing procedure by expert clinicians or technicians to ensure correctness, after which they are designated as ground truth masks.
Figure 1. Overview of the FIREFLY GT construction process.
[1] Kyoung Jin Noh, Sang Jun Park, Soochahn Lee:
Fine-scale vessel extraction in fundus images by registration with fluorescein angiography.
In Proc. International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI) Book 1: 779-787 (2019)
[2] Kyoung Jin Noh, Jooyoung Kim, Sang Jun Park, Soochahn Lee:
Multimodal Registration of Fundus Images With Fluorescein Angiography for Fine-Scale Vessel Segmentation.
IEEE Access 8: 63757-63769 (2020)
[3] Kyoung Jin Noh, Sang Jun Park, Soochahn Lee:
Combining Fundus Images and Fluorescein Angiography for Artery/Vein Classification Using the Hierarchical Vessel Graph Network.
In Proc. International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI) 595-605 (2020)
[link][pdf][presentation][slides][code]
[4] Sojung Go, Jooyoung Kim, Kyoung Jin Noh, Sang Jun Park, Soochahn Lee
Combined Deep Learning of Fundus Images and Fluorescein Angiography for Retinal Artery/Vein Classification.
IEEE Access 10, 70688-70698 (2022)
[link]
The FIREFLY-Gen dataset
The FIREFLY-Gen dataset [5] is a synthetic dataset, comprising images generated by a combined framework of a diffusion model, a GAN, and super-resolution, trained on the FIREFLY dataset. The diffusion model is trained to generate artery/vein masks to create the vascular structure, which is applied to condition a pix2pix GAN to produce retinal fundus images. As we have only applied DDPM [6,7] based diffusion, super-resolution is performed to enhance the resolution of the generated A/V mask and fundus images using the EDSR method [8].
By generating the A/V masks and conditioning fundus image generation on the masks, structurally realistic retinal fundus images are generated. Furthermore, the FIREFLY-Gen can be distribution without privacy concerns, and have been found to be effective as data augmentation for vessel segmentation and A/V classification [5].
Figure 2. Overview of the generative framework to for FIREFLY-Gen.
Figure 3. (Left) A real fundus image with GT A/V mask sample from FIREFLY dataset. (Right) Sample of generated fundus image and artery/vein mask from the proposed method.
[5] Sojung Go, Younghoon Ji, Sang Jun Park, Soochahn Lee:
Generation of Structurally Realistic Retinal Fundus Images with Diffusion Models.
In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2024, pp. 2335-2344
[link]
[6] Ho, Jonathan, Ajay Jain, and Pieter Abbeel. "Denoising diffusion probabilistic models." Advances in neural information processing systems 33 (2020): 6840-6851.
[7] Dhariwal, Prafulla, and Alexander Nichol. "Diffusion models beat gans on image synthesis." Advances in neural information processing systems 34 (2021): 8780-8794.
[8] Lim, Bee, Sanghyun Son, Heewon Kim, Seungjun Nah, and Kyoung Mu Lee. "Enhanced deep residual networks for single image super-resolution." In Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp. 136-144. 2017.
To download the FIREFLY-Gen dataset, please fill the following form. A download link will be sent via the e-mail provided address.