提出无需额外头的原生分割ViT，让模型自动生成分割掩码。
Native Segmentation Vision Transformers
- 用内容感知分组层动态聚合图像特征，替代传统下采样。
- 仅靠分组层即可生成强分割掩码，零样本效果优异。
- 适合追求轻量高效分割模型的研究者和开发者。
统一下采样仍是视觉骨干网络降低空间分辨率的默认方法。本文提出一种新设计，基于内容感知的空间分组层，根据图像边界和语义内容动态将标记（tokens）归并为更少的集合。在连续骨干阶段堆叠该分组层，可自然生成层次化分割结果，形成我们提出的原生分割视觉变换器（Native Segmentation Vision Transformer）。我们证明，通过精心设计架构，仅依靠分组层即可涌现出强大的分割掩码，无需额外的分割专用头部。这为一种新的原生、骨干级分割范式奠定了基础，实现了无需掩码监督的强零样本性能，同时提供一种极简高效的下游分割模型设计。项目主页：https://research.nvidia.com/labs/dvl/projects/native-segmentation。
原文摘要 · Abstract (English)
Uniform downsampling remains the de facto standard for reducing spatial resolution in vision backbones. In this work, we propose an alternative design built around a content-aware spatial grouping layer, that dynamically assigns tokens to a reduced set based on image boundaries and their semantic content. Stacking our grouping layer across consecutive backbone stages results in hierarchical segmentation that arises natively in the feature extraction process, resulting in our coined Native Segmentation Vision Transformer. We show that a careful design of our architecture enables the emergence of strong segmentation masks solely from grouping layers, that is, without additional segmentation-specific heads. This sets the foundation for a new paradigm of native, backbone-level segmentation, which enables strong zero-shot results without mask supervision, as well as a minimal and efficient standalone model design for downstream segmentation tasks. Our project page is https://research.nvidia.com/labs/dvl/projects/native-segmentation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。