用轻量解码器让DINO-v3高效做医学与自然图像分割
SegDINO: An Efficient Design for Medical and Natural Image Segmentation with DINO-V3
- 冻结DINO-v3主干，用轻量MLP头直接生成分割图
- 在6个数据集上均达当前最佳，参数少、计算省
- 适合追求高精度且资源受限的医学图像分割场景
自监督视觉模型DINO系列具备出色迁移能力，但将其表征有效适配到分割任务仍具挑战。现有方法常依赖多尺度融合或复杂上采样结构的重型解码器，带来显著参数开销与计算成本。本文提出SegDINO，一种高效分割框架：将预训练的DINOv3主干冻结，搭配轻量解码器。该框架从编码器提取多层级特征，统一至相同分辨率与通道数，并通过轻量MLP头直接预测分割掩码。此设计在保持基础特征表达力的同时，极大减少可训练参数。在六个基准测试中，包括三个医学数据集（TN3K、Kvasir-SEG、ISIC）和三个自然图像数据集（MSD、VMD-D、ViSha），SegDINO持续达到当前最优性能。代码已开源。
原文摘要 · Abstract (English)
The DINO family of self-supervised vision models has shown remarkable transferability, yet effectively adapting their representations for segmentation remains challenging. Existing approaches often rely on heavy decoders with multi-scale fusion or complex upsampling, which introduce substantial parameter overhead and computational cost. In this work, we propose SegDINO, an efficient segmentation framework that couples a frozen DINOv3 backbone with a lightweight decoder. SegDINO extracts multi-level features from the pretrained encoder, aligns them to a common resolution and channel width, and utilizes a lightweight MLP head to directly predict segmentation masks. This design minimizes trainable parameters while preserving the representational power of foundation features. Extensive experiments across six benchmarks, including three medical datasets (TN3K, Kvasir-SEG, ISIC) and three natural image datasets (MSD, VMD-D, ViSha), demonstrate that SegDINO consistently achieves state-of-the-art performance compared to existing methods. Code is available at https://github.com/script-Yang/SegDINO.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。