通过分层表征实现零样本3D异常检测,兼顾视觉与空间异常。
PointAD+: Learning Hierarchical Representations for Zero-shot 3D Anomaly Detection
- 融合点-像素对应关系,构建隐式与显式3D异常表征。
- 在多个未见类别上达到领先性能,提升对复杂异常的理解。
- 适合做跨类别3D异常检测的研究者和工业场景应用。
本文旨在将CLIP在2D上的强泛化能力迁移至3D异常检测,以识别具有高度多样语义类别的未知物体中的异常。为此,提出统一框架,利用点级与像素级信息进行3D异常检测与分割。首先设计PointAD,通过点-像素对应关系,基于渲染像素表示来隐式表征3D异常;随后提出PointAD+,引入显式3D表示,强调空间异常以发现异常的空间关系。提出G-聚合机制,融合几何信息使点表示具备空间感知能力。通过层次化表征学习,将隐式(渲染)与显式(几何)异常语义融入分层文本提示,并引入跨层级对比对齐,促进两层间异常信息的交互与学习。最终整合双层异常语义,实现对异常的泛化理解。测试阶段可插件式集成RGB信息,进一步提升检测性能。大量实验表明,该方法在多个未见对象中均表现优异,实现了对异常的全面理解。
原文摘要 · Abstract (English)
In this paper, we aim to transfer CLIP's robust 2D generalization capabilities to identify 3D anomalies across unseen objects of highly diverse class semantics. To this end, we propose a unified framework to comprehensively detect and segment 3D anomalies by leveraging both point- and pixel-level information. We first design PointAD, which leverages point-pixel correspondence to represent 3D anomalies through their associated rendering pixel representations. This approach is referred to as implicit 3D representation, as it focuses solely on rendering pixel anomalies but neglects the inherent spatial relationships within point clouds. Then, we propose PointAD+ to further broaden the interpretation of 3D anomalies by introducing explicit 3D representation, emphasizing spatial abnormality to uncover abnormal spatial relationships. Hence, we propose G-aggregation to involve geometry information to enable the aggregated point representations spatially aware. To simultaneously capture rendering and spatial abnormality, PointAD+ proposes hierarchical representation learning, incorporating implicit and explicit anomaly semantics into hierarchical text prompts: rendering prompts for the rendering layer and geometry prompts for the geometry layer. A cross-hierarchy contrastive alignment is further introduced to promote the interaction between the rendering and geometry layers, facilitating mutual anomaly learning. Finally, PointAD+ integrates anomaly semantics from both layers to capture the generalized anomaly semantics. During the test, PointAD+ can integrate RGB information in a plug-and-play manner and further improve its detection performance. Extensive experiments demonstrate the superiority of PointAD+ in ZS 3D anomaly detection across unseen objects with highly diverse class semantics, achieving a holistic understanding of abnormality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。