arXiv:2605.05850cs.CV2026-05

用图像语义指导3D异常检测,解决跨模态差异问题。

Align3D-AD: Cross-Modal Feature Alignment and Dual-Prompt Learning for Zero-shot 3D Anomaly Detection

论文配图:Align3D-AD: Cross-Modal Feature Alignment and Dual-Prompt Learning for Zero-shot 3D Anomaly Detection
图 1 · 摘自论文原文
  • 通过跨模态对齐将3D渲染特征映射到图像语义空间。
  • 在MVTec3D-AD等数据集上优于现有方法,提升零样本泛化能力。
  • 适合关注3D异常检测与多模态融合的研究者。

零样本3D异常检测旨在不依赖目标类别训练数据的情况下识别异常。然而,现有方法主要将3D观测投影为多视角表示,仅捕捉几何信息而缺乏真实视觉语义,并使用在RGB数据上预训练的视觉编码器处理,导致编码器与投影表示间存在显著域差距。为此,我们提出Align3D-AD,一种统一的两阶段框架,利用辅助类别的RGB模态作为跨模态引导进行零样本3D异常检测。首先,引入跨模态特征对齐范式,将渲染特征映射至RGB语义空间,实现直接语义迁移;进一步提出语义一致性重加权策略,根据整体语义一致性重加权局部区域以优化对齐。其次,提出模态感知的双提示对比对齐学习框架,为RGB对齐和渲染特征分配独立提示,捕获模态间互补语义,对比对齐增强提示表示以提高判别力。在MVTec3D-AD、Eyecandies和Real3D-AD上的大量实验表明,Align3D-AD在一对一和跨数据集设置下均持续优于现有方法,验证其泛化能力与鲁棒性。代码与数据集将在论文录用后公开。

原文摘要 · Abstract (English)

Zero-shot 3D anomaly detection aims to identify anomalies without access to training data from target categories. However, existing methods mainly rely on projecting 3D observations into multi-view representations that primarily capture geometric cues rather than realistic visual semantics and process them with vision encoders pretrained on RGB data, leading to a significant domain gap between the encoder and the projected representations. To address this issue, we propose Align3D-AD, a unified two-stage framework that leverages the RGB modality from auxiliary categories as cross-modal guidance for zero-shot 3D anomaly detection. First, we introduce a cross-modal feature alignment paradigm that maps rendering features into the RGB semantic space. Unlike prior works that implicitly rely on pretrained encoders, our method enables direct semantic transfer from RGB observations. A semantic consistency reweighting strategy is further introduced to refine feature alignment by reweighting local regions according to holistic semantic consistency. Second, we propose a modality-aware prompt learning framework with dual-prompt contrastive alignment. By assigning independent prompts to RGB-aligned and rendering features, our method captures complementary semantics across modalities, while the contrastive alignment further enhances prompt representations to improve discriminability. Extensive experiments on MVTec3D-AD, Eyecandies, and Real3D-AD demonstrate that Align3D-AD consistently outperforms existing zero-shot methods under both one-vs-rest and cross-dataset settings, highlighting its generalization capability and robustness. Code and the dataset will be made available once our paper is accepted.

3D异常检测跨模态对齐零样本学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。