Joint weakly and semi-supervised deep learning for localization and classification of masses in breast ultrasound images.

Authors: Seung Yeon Shin, Soochahn Lee, Il Dong Yun, Sun Mi Kim, Kyoung Mu Lee

TL;DR: A joint weakly and semi-supervised deep learning framework for localizing and classifying breast masses in ultrasound images, reducing the annotation burden while maintaining strong performance.

What this paper is about

Example of breast ultrasound images with masses. Bounding boxes with solid and dashed lines respectively represent ground truths (GT) and detections using the proposed method. Boxes are colored as blue (red) if the GT or predicted label is benign (malignant).

Getting pixel-level annotations for medical images is expensive and time-consuming, especially in breast ultrasound where masses need to be precisely delineated. This creates a bottleneck for training deep learning models. The question is how to train accurate localization and classification models when only a mix of weak labels (image-level) and limited pixel-level annotations are available.

Key idea

Illustration of the proposed framework. (a) Images from two different data streams are forward-propagated into a shared network. (b) The Faster R-CNN [34] used for the “network” of (a). The network is composed of the region proposal network (RPN) and Fast R-CNN [35] with shared convolutional layers. This figure was previously presented in [34] and is reprinted in this paper for the description of the Faster R-CNN. We also note that the proposed method is a general framework; hence, other supervised approaches can also be adopted. (c) An image-level loss is used for images from DX, whereas region-level losses are used for images from DXLoc.

The framework combines weakly supervised learning (using image-level labels indicating whether a mass is present) with semi-supervised learning (using a small set of fully annotated images) in a unified training pipeline. This joint approach leverages both types of supervision to learn mass localization and classification more effectively than either strategy alone.

Why it matters

Qualitative results on the SNUBH dataset. Each row shows different images. Each of the top three rows presents a case with various types of masses, which can be small, large, or unclear. The bottom two rows present failure cases where either localization or classification fails. Bounding boxes with solid and dashed lines respectively represent ground truths (GT) and detections using the proposed method. Boxes are colored as blue (red) if the GT or predicted label is benign (malignant). Figure best viewed in color.

Published in IEEE Transactions on Medical Imaging, this work significantly reduces the annotation effort needed for training breast ultrasound CAD systems, making deep learning more practical for clinical deployment in breast cancer screening.