让视觉模型自动发现物体部件，像人一样识别鸟的头翅膀
RATS! Patches Talk Through Registers: Emergent Parts in Register Attention Transformers
- 用可学习的寄存器分解分类令牌，通过三步注意力机制路由图像块信息
- 在五个分割基准上平均提升12 mIoU，ADE20K和COCO也显著领先
- 无需标注就能自发形成语义部件，适合需要可解释视觉表征的研究
当人类看到一只鸟时，不仅知道是‘鸟’，还能识别出头、翅膀、爪子等结构化部件，这些部件可跨实例复用。我们探究自监督视觉模型能否自主发现这种组合结构。为此提出RATS（Register Attention Transformers），将分类令牌分解为N个可学习的寄存器令牌，通过L→N→N→L的压缩-通信-广播注意力路径路由图像块信息。这N个寄存器分布在H个注意力头中，不同头的寄存器互不交互。无辅助损失或部件标注下，每个寄存器自发特化为原型语义区域，其结构类似物体部件。RATS在五个分割基准上平均提升12 mIoU，其中ADE20K提升1.11 mIoU，COCO提升0.2 AP^m。其寄存器字典还表现出跨类别相关类别的部件级一致性与语义相近性。结果表明，RATS可能为结构化、可解释的视觉表示学习提供有效架构先验。
原文摘要 · Abstract (English)
When humans see a bird, they recognize far more than just "bird" -- they see a head, wings, and talons, a structured assembly of reusable parts that can be identified across every bird they have ever seen. We ask whether a self-supervised visual model can discover the same compositional structure on its own. To this end, we propose RATS (Register Attention Transformers), which decomposes the classification token into N learnable register tokens that route patch information through an L->N->N->L bottleneck via a three-step compress-communicate-broadcast attention. The N registers are partitioned across the H attention heads, so that registers assigned to different heads do not interact with each other. Without auxiliary losses or part annotations, each register spontaneously specializes into a proto-semantic region whose emerging structure resembles object parts. RATS surpasses all baselines by +12 mIoU on average across five segmentation benchmarks, with consistent gains on ADE20K (+1.11 mIoU) and COCO (+0.2 AP^m). Its register dictionary further exhibits part-level consistency and semantic proximity across related categories. Our results suggest that RATS may provide a useful architectural prior for structured and interpretable visual representation learning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。