不重建图像特征，直接在令牌空间预测分割掩码，提升效率。
Token-Space Mask Prediction for Efficient Vision Transformer Segmentation
- 在令牌空间计算掩码得分，避免图像空间重构
- 相比以往方法降低计算与内存开销，速度提升明显
- 适合嵌入式视觉系统部署，尤其移动端
基于查询的视觉 Transformer 分割模型通常通过重建密集的空间特征图来预测掩码，沿用了卷积架构的设计模式。本文表明，这种显式的图像空间重构并非必需。我们提出 TokenMask，一种在令牌空间中直接从查询令牌相似性计算掩码得分的头结构，并在得分空间而非特征空间进行插值。该设计保留了原有的线性评分机制，同时简化了计算结构。在多种 ViT 骨干网络、数据集和分割任务上，TokenMask 均显著优于先前方法，在保持竞争力准确率的同时，降低了计算与内存需求，在 NVIDIA Jetson AGX Orin 上使用 TensorRT FP16 推理时实现了明显的加速。总体而言，TokenMask 为嵌入式视觉系统提供了更简洁、更易部署的设计。
原文摘要 · Abstract (English)
Query-based Vision Transformer segmentation models typically reconstruct dense spatial feature maps to predict masks, inheriting design patterns from convolutional architectures. We show that this explicit image-space reconstruction is not required. We introduce TokenMask, a token-space mask head that computes mask logits directly from query-token affinities and performs interpolation in logit space rather than feature space. This reformulation preserves the original linear scoring mechanism while simplifying the computational structure. Across diverse ViT backbones, datasets and segmentation tasks, TokenMask consistently improves efficiency over prior approaches by reducing computational and memory requirements while maintaining competitive accuracy, leading to tangible speedups on NVIDIA Jetson AGX Orin using TensorRT FP16 inference. Overall, TokenMask yields a simpler and more deployment-friendly design for embedded vision systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。