Research

Our research spans multimodal learning, medical image analysis, and generative modeling — with a consistent focus on building methods that are anatomically grounded, annotation-efficient, and deployable in real clinical and real-world settings. Select a topic below for an overview and the associated publications.

Analysis of Multimodal LLMs

Benchmarks and diagnostic tools for evaluating how multimodal large language models integrate audio, visual, and linguistic signals — including audiovisual speech understanding, active speaker detection, and contamination detection.

Registration of Multimodal Retinal Images

Aligning fundus, fluorescein angiography, and ultra-widefield retinal images across large appearance and geometric gaps, using score-based diffusion, autoregressive transformation, and generative enhancement.

The FIREFLY Dataset

A registered fundus–FA dataset with fine-scale vessel and artery/vein annotations, extended with diffusion-based synthetic counterparts (FIREFLY-Gen).

Vessel Analysis of Retinal Images

Topology-aware vessel segmentation and artery/vein classification, interactive and prompt-based refinement, fundus image enhancement, and generative modeling for vessel-annotated data.

Mass and Abnormality Detection from Chest X-ray

Generative-comparison anomaly detection and disentangled targeted generation for chest X-ray analysis under weak or no lesion-level supervision.

Mass Detection from Mammography and Ultrasound

Cascade and classification-based microcalcification detection in mammograms, and weakly/semi-supervised mass detection in breast ultrasound.

Knee Cartilage Segmentation from MR Images

3D graph-cut segmentation, localized shape and appearance modeling, hierarchical MRFs with localized classifiers, and unified automatic/interactive segmentation frameworks — evaluated on the Osteoarthritis Initiative (OAI).