Fully leveraging deep learning methods for constructing retinal fundus photomontages.

Authors: Jooyoung Kim, Sojung Go, Kyoung Jin Noh, Sang Jun Park, Soochahn Lee

TL;DR: A fully deep-learning-based pipeline for automatically stitching multiple retinal fundus photographs into a single wide-field photomontage, replacing traditional feature-matching heuristics with learned representations.

What this paper is about

Retinal fundus cameras capture only a limited field of view, so clinicians often need to mentally piece together several snapshots to see the full retina. Building a seamless photomontage from these narrow-field images has traditionally relied on hand-crafted feature matching, which struggles with the low-contrast, repetitive textures of the retina.

Key idea

Overview of the proposed framework for constructing retinal fundus photomontages. Through deep learning based object detection, we are able to apply prior knowledge of the fovea and optic disc to determine the optimal order in which to integrate the images into the montage. Deep learning is also leveraged to reduce errors in registration.

A visual summary of the frame integration pipeline, including a two-step rigid and non-rigid registration method adapted from our MICCAI 2019 paper, together with image blending.

This work replaces the conventional montage construction pipeline with end-to-end deep learning methods. The approach leverages deep networks for both image registration (aligning overlapping fundus photos) and blending (stitching them into a single coherent mosaic), fully automating the process without manual landmark selection.

Why it matters

Automated wide-field montages give ophthalmologists a comprehensive view of the retina in one image, improving diagnostic efficiency for diseases like diabetic retinopathy and retinal detachment – especially in clinics that lack expensive ultra-widefield cameras.