从架构角度重探多模态融合，提升3D异常检测性能
Revisiting Multimodal Fusion for 3D Anomaly Detection from an Architectural Perspective
- 分析模块内与模块间融合架构对检测效果的影响
- 提出3D-ADNAS，联合搜索融合策略与模态专用模块
- 在多种模型规模下实现精度、速度、内存的全面提升
现有研究主要聚焦于设计更高效的多模态融合策略，但较少关注多模态融合架构（拓扑）设计对3D异常检测（3D-AD）的贡献。本文系统研究融合架构设计对3D-AD的影响，涵盖模块内融合（如早期、中期、晚期特征融合）和模块间融合策略。通过理论与实验分析，揭示架构设计的关键作用，并首次将主流神经架构搜索（NAS）范式拓展至3D-AD场景，提出3D-ADNAS，同步搜索多模态融合策略与模态专用模块。大量实验证明，3D-ADNAS在不同模型容量下均显著提升3D-AD的准确性、帧率与内存效率，且在少样本3D-AD任务中展现巨大潜力。
原文摘要 · Abstract (English)
Existing efforts to boost multimodal fusion of 3D anomaly detection (3D-AD) primarily concentrate on devising more effective multimodal fusion strategies. However, little attention was devoted to analyzing the role of multimodal fusion architecture (topology) design in contributing to 3D-AD. In this paper, we aim to bridge this gap and present a systematic study on the impact of multimodal fusion architecture design on 3D-AD. This work considers the multimodal fusion architecture design at the intra-module fusion level, i.e., independent modality-specific modules, involving early, middle or late multimodal features with specific fusion operations, and also at the inter-module fusion level, i.e., the strategies to fuse those modules. In both cases, we first derive insights through theoretically and experimentally exploring how architectural designs influence 3D-AD. Then, we extend SOTA neural architecture search (NAS) paradigm and propose 3D-ADNAS to simultaneously search across multimodal fusion strategies and modality-specific modules for the first time.Extensive experiments show that 3D-ADNAS obtains consistent improvements in 3D-AD across various model capacities in terms of accuracy, frame rate, and memory usage, and it exhibits great potential in dealing with few-shot 3D-AD tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。