利用无标签样本的相似性,实现工业缺陷零样本分类与分割
MuSc-V2: Zero-Shot Multimodal Industrial Anomaly Classification and Segmentation with Mutual Scoring of Unlabeled Samples
- 通过互评机制挖掘无标签样本间的相似性特征
- 在MVTec 3D-AD上提升23.7%的异常检测准确率
- 支持多模态输入,适合快速适配不同产品线
零样本异常分类(AC)与分割(AS)旨在无需标注样本的情况下识别和定位缺陷。本文揭示了一个被忽视的关键特性:工业产品中的正常图像块通常在2D外观和3D形状上均存在大量相似块,而异常则表现多样且孤立。为此,提出互评框架MuSc-V2,灵活支持单模态(2D/3D)或多模态输入。首先通过迭代点分组(IPG)优化3D表示,减少由表面不连续导致的误检;接着采用多度相似邻域聚合(SNAMD)融合2D/3D邻域信息,生成更具判别力的多尺度块特征用于互评。核心为互评机制(MSM),使各模态样本相互打分,并引入跨模态异常增强(CAE)融合双模态分数以恢复模态缺失异常。最后,受限邻域重评分(RsCon)基于更典型样本抑制误分类。该框架在全数据集及子集上均表现稳健,具备良好可扩展性。实验表明,MuSc-V2在MVTec 3D-AD数据集上获得+23.7%的AP提升,在Eyecandies数据集上提升+19.3%,超越多数零样本方法,甚至优于部分少样本方法。
原文摘要 · Abstract (English)
Zero-shot anomaly classification (AC) and segmentation (AS) methods aim to identify and outline defects without using any labeled samples. In this paper, we reveal a key property that is overlooked by existing methods: normal image patches across industrial products typically find many other similar patches, not only in 2D appearance but also in 3D shapes, while anomalies remain diverse and isolated. To explicitly leverage this discriminative property, we propose a Mutual Scoring framework (MuSc-V2) for zero-shot AC/AS, which flexibly supports single 2D/3D or multimodality. Specifically, our method begins by improving 3D representation through Iterative Point Grouping (IPG), which reduces false positives from discontinuous surfaces. Then we use Similarity Neighborhood Aggregation with Multi-Degrees (SNAMD) to fuse 2D/3D neighborhood cues into more discriminative multi-scale patch features for mutual scoring. The core comprises a Mutual Scoring Mechanism (MSM) that lets samples within each modality to assign score to each other, and Cross-modal Anomaly Enhancement (CAE) that fuses 2D and 3D scores to recover modality-specific missing anomalies. Finally, Re-scoring with Constrained Neighborhood (RsCon) suppresses false classification based on similarity to more representative samples. Our framework flexibly works on both the full dataset and smaller subsets with consistently robust performance, ensuring seamless adaptability across diverse product lines. In aid of the novel framework, MuSc-V2 achieves significant performance improvements: a $\textbf{+23.7\%}$ AP gain on the MVTec 3D-AD dataset and a $\textbf{+19.3\%}$ boost on the Eyecandies dataset, surpassing previous zero-shot benchmarks and even outperforming most few-shot methods. The code will be available at The code will be available at \href{https://github.com/HUST-SLOW/MuSc-V2}{https://github.com/HUST-SLOW/MuSc-V2}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。