通过动态分配高分辨率标记，高效提取图像语义特征。
ARTA: Adaptive Mixed-Resolution Token Allocation for Efficient Dense Feature Extraction
- 从低分辨率标记开始，按语义边界需求动态添加高分辨率标记。
- 在ADE20K上达54.6 mIoU，参数量约100M但计算量更低。
- 适合需要高效密集特征提取的视觉任务，如场景分割。
我们提出ARTA，一种用于高效密集特征提取的混合分辨率自适应视觉变换器。不同于从高分辨率（精细）标记开始的模型，ARTA起始于低分辨率（粗糙）标记，并使用轻量级分配器预测哪些区域需要更多精细标记。该分配器迭代地预测语义（类别）边界得分，并将额外标记分配给高于低阈值的补丁，使标记密度集中在边界附近，同时保持对弱边界证据的高敏感性。这种定向分配促使标记代表单一语义类别而非混合类别。混合分辨率注意力实现粗细标记间的交互，将计算聚焦于语义复杂区域，避免在均质区域进行冗余处理。实验表明，ARTA在ADE20K和COCO-Stuff上达到顶尖性能，且显著减少浮点运算次数；在Cityscapes上以极低计算开销实现竞争力表现。例如，ARTA-Base在约100M参数量下于ADE20K取得54.6 mIoU，计算量和内存占用均低于同类主干网络。
原文摘要 · Abstract (English)
We present ARTA, a mixed-resolution coarse-to-fine vision transformer for efficient dense feature extraction. Unlike models that begin with dense high-resolution (fine) tokens, ARTA starts with low-resolution (coarse) tokens and uses a lightweight allocator to predict which regions require more fine tokens. The allocator iteratively predicts a semantic (class) boundary score and allocates additional tokens to patches above a low threshold, concentrating token density near boundaries while maintaining high sensitivity to weak boundary evidence. This targeted allocation encourages tokens to represent a single semantic class rather than a mixture of classes. Mixed-resolution attention enables interaction between coarse and fine tokens, focusing computation on semantically complex areas while avoiding redundant processing in homogeneous regions. Experiments demonstrate that ARTA achieves state-of-the-art results on ADE20K and COCO-Stuff with substantially fewer FLOPs, and delivers competitive performance on Cityscapes at markedly lower compute. For example, ARTA-Base attains 54.6 mIoU on ADE20K in the ~100M-parameter class while using fewer FLOPs and less memory than comparable backbones.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。