arXiv:2510.21171cs.CV2025-10被引 1

让图像每个局部区域匹配专属文本空间,提升零样本异常检测精度

TokenCLIP: Token-wise Prompt Learning for Zero-shot Anomaly Detection

  • 为每个视觉令牌动态分配定制化文本子空间,实现细粒度对齐
  • 通过最优传输建模语义匹配,使不同区域聚焦不同异常特征
  • 适合需要精准定位异常的工业质检场景

将CLIP用于未见物体的零样本异常检测展现出强大潜力。然而,现有方法通常依赖单一文本空间与多样对象和领域进行对齐,这种无差别对齐限制了模型对异质异常语义的捕捉能力。我们提出TokenCLIP,一种基于令牌级别的自适应框架,实现视觉与可学习文本空间间的动态对齐,支持细粒度异常学习。不同于将所有视觉令牌映射到统一、不可区分的文本空间,TokenCLIP为每个令牌分配一个反映其视觉特性的定制化文本子空间。显式为每个令牌配置独立可学习空间计算成本过高且易优化不足。因此,我们扩展原始文本空间为一组正交子空间,并根据语义相似性动态分配每个令牌至子空间组合,兼顾定制化与效率。我们将动态对齐建模为最优传输问题,依据语义相似性将图像中所有视觉令牌运输至文本子空间,传输约束确保子空间间充分优化并专注不同语义。求解后得到的传输方案自适应地将每个令牌分配至语义相关子空间。随后采用top-k掩码稀疏化该方案,使子空间专用于不同视觉区域。大量实验验证了TokenCLIP的优越性。

原文摘要 · Abstract (English)

Adapting CLIP for anomaly detection on unseen objects has shown strong potential in a zero-shot manner. However, existing methods typically rely on a single textual space to align with visual semantics across diverse objects and domains. The indiscriminate alignment hinders the model from accurately capturing varied anomaly semantics. We propose TokenCLIP, a token-wise adaptation framework that enables dynamic alignment between visual and learnable textual spaces for fine-grained anomaly learning. Rather than mapping all visual tokens to a single, token-agnostic textual space, TokenCLIP aligns each token with a customized textual subspace that represents its visual characteristics. Explicitly assigning a unique learnable textual space to each token is computationally intractable and prone to insufficient optimization. We instead expand the token-agnostic textual space into a set of orthogonal subspaces, and then dynamically assign each token to a subspace combination guided by semantic affinity, which jointly supports customized and efficient token-wise adaptation. To this end, we formulate dynamic alignment as an optimal transport problem, where all visual tokens in an image are transported to textual subspaces based on semantic similarity. The transport constraints of OT ensure sufficient optimization across subspaces and encourage them to focus on different semantics. Solving the problem yields a transport plan that adaptively assigns each token to semantically relevant subspaces. A top-k masking is then applied to sparsify the plan and specialize subspaces for distinct visual regions. Extensive experiments demonstrate the superiority of TokenCLIP.

异常检测CLIP零样本动态对齐

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。