arXiv:2505.06333cs.LGcs.AI2025-05IJCAI被引 6

融合时序与图像数据,提升装配线异常检测的鲁棒性与可解释性。

NSF-MAP: Neurosymbolic Multimodal Fusion for Robust and Interpretable Anomaly Prediction in Assembly Pipelines

  • 采用决策层融合策略整合时序与图像信息。
  • 结合迁移学习与知识注入,性能优于传统基线方法。
  • 适合工业质检场景,兼顾准确性与结果可解释性。

在现代装配流水线中,异常检测对保障产品质量和运营效率至关重要。传统单模态方法难以捕捉复杂预测环境中多模态数据间的细微关联。本文提出一种基于神经符号AI与融合的多模态异常预测方法,构建了基于时间序列与图像的决策层融合模型。研究引入三大创新:时序与图像的决策层融合建模、融合中的迁移学习,以及知识注入学习。通过自建并公开的多模态数据集进行评估,并开展全面消融实验,验证预处理技术与融合模型的有效性。结果表明,基于神经符号AI与迁移学习的融合方法能有效利用时序与图像数据的互补优势,在装配线异常预测中实现更强鲁棒性与可解释性。代码、数据集、补充材料及演示见https://github.com/ChathurangiShyalika/NSF-MAP。

原文摘要 · Abstract (English)

In modern assembly pipelines, identifying anomalies is crucial in ensuring product quality and operational efficiency. Conventional single-modality methods fail to capture the intricate relationships required for precise anomaly prediction in complex predictive environments with abundant data and multiple modalities. This paper proposes a neurosymbolic AI and fusion-based approach for multimodal anomaly prediction in assembly pipelines. We introduce a time series and image-based fusion model that leverages decision-level fusion techniques. Our research builds upon three primary novel approaches in multimodal learning: time series and image-based decision-level fusion modeling, transfer learning for fusion, and knowledge-infused learning. We evaluate the novel method using our derived and publicly available multimodal dataset and conduct comprehensive ablation studies to assess the impact of our preprocessing techniques and fusion model compared to traditional baselines. The results demonstrate that a neurosymbolic AI-based fusion approach that uses transfer learning can effectively harness the complementary strengths of time series and image data, offering a robust and interpretable approach for anomaly prediction in assembly pipelines with enhanced performance. \noindent The datasets, codes to reproduce the results, supplementary materials, and demo are available at https://github.com/ChathurangiShyalika/NSF-MAP.

异常检测多模态融合工业AI可解释性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。