arXiv:2501.04784cs.CVcs.LG2025-01中稿 · ICASSP 2025被引 4

通过融合特殊标记与寄存器信息,提升ViT在分布外场景下的泛化与异常检测能力。

Leveraging Registers in Vision Transformers for Robust Adaptation

  • 引入寄存器令牌,分离高范数图像块并保留全局特征
  • 结合CLS与平均池化寄存器嵌入,使分布外准确率提升2-4%
  • 无需额外计算开销,适合部署于实际异常检测系统

视觉变换器(ViTs)因其捕捉全局图像表示的能力而在多种任务中表现优异。近期研究发现,ViTs中存在高范数令牌,会干扰无监督目标发现。为此,提出使用“寄存器”——额外令牌,用于隔离高范数的图像块令牌,同时捕获图像级全局信息。尽管寄存器在目标发现中已被广泛研究,其在分布外(OOD)场景下的泛化能力仍不明确。本文考察寄存器嵌入在增强泛化与异常排斥方面的效用,提出一种简单方法:将ViT中常用的CLS令牌嵌入与平均池化的寄存器嵌入相结合,生成下游分类器使用的特征表示。实验表明,该方法显著提升分布外性能,使顶级1准确率提高2-4%,异常检测误报率降低2-3%,且保持分布内性能不变。重要的是,这些改进无需额外计算开销,适用于多种训练有无寄存器的ViT主干网络。

原文摘要 · Abstract (English)

Vision Transformers (ViTs) have shown success across a variety of tasks due to their ability to capture global image representations. Recent studies have identified the existence of high-norm tokens in ViTs, which can interfere with unsupervised object discovery. To address this, the use of "registers" which are additional tokens that isolate high norm patch tokens while capturing global image-level information has been proposed. While registers have been studied extensively for object discovery, their generalization properties particularly in out-of-distribution (OOD) scenarios, remains underexplored. In this paper, we examine the utility of register token embeddings in providing additional features for improving generalization and anomaly rejection. To that end, we propose a simple method that combines the special CLS token embedding commonly employed in ViTs with the average-pooled register embeddings to create feature representations which are subsequently used for training a downstream classifier. We find that this enhances OOD generalization and anomaly rejection, while maintaining in-distribution (ID) performance. Extensive experiments across multiple ViT backbones trained with and without registers reveal consistent improvements of 2-4\% in top-1 OOD accuracy and a 2-3\% reduction in false positive rates for anomaly detection. Importantly, these gains are achieved without additional computational overhead.

视觉变换器异常检测分布外泛化寄存器

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。