用细粒度描述和可变形定位实现零样本异常检测,适应新场景更快更准。
FiLo++: Zero-/Few-Shot Anomaly Detection by Fused Fine-Grained Descriptions and Deformable Localization
- 用大语言模型生成每类物体的专属异常描述,提升文本匹配精度。
- 结合多尺度可变形交互模块,精确定位不同形状大小的异常区域。
- 适合冷启动场景,尤其适用于少样本或无样本的工业质检任务。
异常检测方法通常需要大量目标类别的正常样本进行训练,限制了其在需快速适应场景(如冷启动)中的应用。零样本与少样本异常检测无需提前获取目标类别的标注样本,是极具前景的研究方向。现有方法多依赖多模态模型通过图像-文本相似性检测与定位异常,但其手工设计的通用描述难以捕捉各类物体可能出现的多样化异常,且简单的块级图像-文本匹配常难以定位形状尺寸各异的异常区域。为此,本文提出FiLo++方法,包含两个关键组件:第一,融合细粒度描述(FusDes),利用大语言模型为每类物体生成异常描述,结合固定与可学习提示模板,并采用运行时提示过滤机制,生成更准确、任务相关的文本描述;第二,可变形定位(DefLoc),将视觉基础模型Grounding DINO与位置增强文本描述及多尺度可变形跨模态交互(MDCI)模块结合,实现对多种形状与尺寸异常的精准定位。此外,设计了位置增强的块匹配方法以提升少样本检测性能。在多个数据集上的实验表明,FiLo++显著优于现有方法。代码将发布于https://github.com/CASIA-IVA-Lab/FiLo。
原文摘要 · Abstract (English)
Anomaly detection methods typically require extensive normal samples from the target class for training, limiting their applicability in scenarios that require rapid adaptation, such as cold start. Zero-shot and few-shot anomaly detection do not require labeled samples from the target class in advance, making them a promising research direction. Existing zero-shot and few-shot approaches often leverage powerful multimodal models to detect and localize anomalies by comparing image-text similarity. However, their handcrafted generic descriptions fail to capture the diverse range of anomalies that may emerge in different objects, and simple patch-level image-text matching often struggles to localize anomalous regions of varying shapes and sizes. To address these issues, this paper proposes the FiLo++ method, which consists of two key components. The first component, Fused Fine-Grained Descriptions (FusDes), utilizes large language models to generate anomaly descriptions for each object category, combines both fixed and learnable prompt templates and applies a runtime prompt filtering method, producing more accurate and task-specific textual descriptions. The second component, Deformable Localization (DefLoc), integrates the vision foundation model Grounding DINO with position-enhanced text descriptions and a Multi-scale Deformable Cross-modal Interaction (MDCI) module, enabling accurate localization of anomalies with various shapes and sizes. In addition, we design a position-enhanced patch matching approach to improve few-shot anomaly detection performance. Experiments on multiple datasets demonstrate that FiLo++ achieves significant performance improvements compared with existing methods. Code will be available at https://github.com/CASIA-IVA-Lab/FiLo.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。