TMT通过区域自适应迁移性估计，提升视觉Transformer在跨域分割中的表现。
TMT: Cross-domain Semantic Segmentation with Region-adaptive Transferability Estimation
- 按结构与语义相似性动态分块，局部评估各区域迁移能力。
- 在注意力机制中引入迁移性图，使模型聚焦不确定区域。
- 20个跨域场景下均优于现有方法，显著缓解分布偏移影响。
视觉变换器（ViTs）的进展显著提升了语义分割性能，但其在新目标域的适应仍受分布偏移影响，常破坏全局注意力机制。现有全局与补丁级适配方法忽略图像区域间迁移性的空间差异。为此，我们提出可迁移掩码变换器（TMT），一种基于区域自适应的框架，通过迁移性引导增强跨域表示学习。首先，动态将图像划分为结构与语义相似的连贯区域，并在局部层面估计其域迁移性。随后，将区域级迁移性图直接融入ViTs的自注意力机制，使模型自适应地关注迁移性低、语义不确定性高的区域。在20个多样化的跨域设置下进行的广泛实验表明，TMT不仅有效缓解了域偏移导致的性能下降，且持续优于现有方法。
原文摘要 · Abstract (English)
Recent advances in Vision Transformers (ViTs) have significantly advanced semantic segmentation performance. However, their adaptation to new target domains remains challenged by distribution shifts, which often disrupt global attention mechanisms. While existing global and patch-level adaptation methods offer some improvements, they overlook the spatially varying transferability inherent in different image regions. To address this, we propose the Transferable Mask Transformer (TMT), a region-adaptive framework designed to enhance cross-domain representation learning through transferability guidance. First, we dynamically partition the image into coherent regions, grouped by structural and semantic similarity, and estimates their domain transferability at a localized level. Then, we incorporate region-level transferability maps directly into the self-attention mechanism of ViTs, allowing the model to adaptively focus attention on areas with lower transferability and higher semantic uncertainty. Extensive experiments across 20 diverse cross-domain settings demonstrate that TMT not only mitigates the performance degradation typically associated with domain shift but also consistently outperforms existing approaches.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。