用少量可学习令牌实现视觉Transformer多尺度推理，提升超高清图像分割精度。
Adapting Vision Transformers to Ultra-High Resolution Semantic Segmentation with Relay Tokens
- 通过局部与全局双分支并行处理，结合可学习令牌传递特征。
- 在三个超高清数据集上实现最高15%的相对mIoU提升。
- 轻量级设计，参数增加少于2%，可直接嵌入ViT/Swin等模型。
当前超高清图像分割方法要么滑动窗口导致全局上下文丢失，要么下采样损失细节。本文提出一种简单有效的方法，为视觉Transformer引入显式多尺度推理，同时保留局部细节和全局感知。具体而言，将图像在局部尺度（高分辨率、小块）和全局尺度（低分辨率、大块）并行处理，并通过少量可学习的中继令牌在两分支间聚合与传播特征。该设计可直接插入标准Transformer主干（如ViT和Swin），参数增加少于2%。在三个超高清分割基准（Archaeoscape、URUR、Gleason）及常规Cityscapes数据集上的实验表明，性能持续提升，最高达15%相对mIoU增益。代码与预训练模型已公开于https://archaeoscape.ai/work/relay-tokens/。
原文摘要 · Abstract (English)
Current approaches for segmenting ultra high resolution images either slide a window, thereby discarding global context, or downsample and lose fine detail. We propose a simple yet effective method that brings explicit multi scale reasoning to vision transformers, simultaneously preserving local details and global awareness. Concretely, we process each image in parallel at a local scale (high resolution, small crops) and a global scale (low resolution, large crops), and aggregate and propagate features between the two branches with a small set of learnable relay tokens. The design plugs directly into standard transformer backbones (eg ViT and Swin) and adds fewer than 2 % parameters. Extensive experiments on three ultra high resolution segmentation benchmarks, Archaeoscape, URUR, and Gleason, and on the conventional Cityscapes dataset show consistent gains, with up to 15 % relative mIoU improvement. Code and pretrained models are available at https://archaeoscape.ai/work/relay-tokens/ .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。