提出轻量通用掩码适配器，统一评测不同主干网络分割性能。
LUMA: Benchmarking Segmentation via a Lightweight Universal Mask Adapter
- 用轻量跨注意力头统一适配各类主干网络，作为黑箱特征提取器。
- 在ADE20K和Cityscapes上测试20个主干、11种预训练方案，发现纯ViT在各分辨率下吞吐最优。
- 预训练目标比架构更重要，高效令牌混合器在高分辨率下并不高效。
比较图像分割中Transformer主干网络的性能存在混淆：每个主干通常搭配不同的解码器、训练方案和预训练方式，导致报告的差异很少反映主干本身的真实表现。本文提出轻量通用掩码适配器（LUMA），一种与主干无关的轻量级掩码变压器头，将任何主干视为黑箱特征提取器，通过低成本交叉注意力让一组查询读取其特征。LUMA在更低开销下达到EoMT（当前最先进的高效ViT分割器）的精度，且可无缝适配等向、分层、卷积及专家混合型主干。固定该头部，我们在统一现代训练流程下，在ADE20K和Cityscapes数据集上对20个主干、11种预训练策略及多种分辨率进行了基准测试。结果表明，'高效'的令牌混合器即使在推动其发展的高分辨率下也未能实现效率提升，而普通ViT在所有分辨率下均保持吞吐率最优。此外，预训练目标——而非网络结构——才是决定分割质量的关键因素。
原文摘要 · Abstract (English)
Comparing transformer backbones for image segmentation is confounded: each is paired with a different decoder, recipe, and pretraining, so reported differences rarely reflect the backbone itself. We introduce the Lightweight Universal Mask Adapter (LUMA), a lightweight, backbone-agnostic mask-transformer head that treats any backbone as a black-box feature extractor, letting a set of queries read from its features through cheap cross-attention. LUMA matches the accuracy of EoMT, the state-of-the-art efficient ViT-segmenter, at lower cost, while attaching unchanged to isotropic, hierarchical, convolutional, and mixture-of-experts backbones alike. Holding this head fixed, we benchmark 20 backbones, 11 pretraining schemes and a range of resolutions on ADE20K and Cityscapes under one modern recipe. We find that ``efficient'' token mixers fail to deliver efficiency even at the high resolutions that motivate them, with plain ViT holding the throughput Pareto-front at every resolution. Additionally, the pretraining objective, not the architecture, the lever the field has tuned hardest, governs segmentation quality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。