arXiv:2603.25103cs.LGcs.AI2026-03

通过可学习的梯度调控,让多模态模型在传感器失效时仍能准确检测并修复异常。

Layer-Specific Lipschitz Modulation for Fault-Tolerant Multimodal Representation Learning

  • 基于局部故障传播理论,设计分层 Lipschitz 调控机制控制模型敏感度。
  • 在故障数据集上,异常检测准确率与重建质量均显著优于基线方法。
  • 适合工业、自动驾驶等对可靠性要求高的多模态系统应用。

面向工业与安全关键场景中的多模态系统,需在部分传感器失效、信号退化或跨模态不一致下保持可靠性。本文提出一种数学严谨的容错多模态表征学习框架,统一自监督异常检测与错误修正。基于扰动传播的理论分析,推导出基于 Lipschitz 与 Jacobian 的判据,判断神经算子是否放大或抑制局部故障。据此设计两阶段自监督训练:先在干净数据上预训练多模态卷积自编码器以保留潜在空间中的局部异常信号;再引入可学习计算模块(含全连接层),结合修正目标与对比学习目标实现异常识别。进一步提出分层 Lipschitz 调制与梯度裁剪机制,精确控制检测与修正模块的敏感性。在多个多模态故障数据集上的实验表明,该方法在传感器污染条件下显著提升异常检测准确率与重建性能。整体框架实现了理论鲁棒性保证与实际容错学习的融合。

原文摘要 · Abstract (English)

Modern multimodal systems deployed in industrial and safety-critical environments must remain reliable under partial sensor failures, signal degradation, or cross-modal inconsistencies. This work introduces a mathematically grounded framework for fault-tolerant multimodal representation learning that unifies self-supervised anomaly detection and error correction within a single architecture. Building upon a theoretical analysis of perturbation propagation, we derive Lipschitz- and Jacobian-based criteria that determine whether a neural operator amplifies or attenuates localized faults. Guided by this theory, we propose a two-stage self-supervised training scheme: pre-training a multimodal convolutional autoencoder on clean data to preserve localized anomaly signals in the latent space, and expanding it with a learnable compute block composed of dense layers for correction and contrastive objectives for anomaly identification. Furthermore, we introduce layer-specific Lipschitz modulation and gradient clipping as principled mechanisms to control sensitivity across detection and correction modules. Experimental results on multimodal fault datasets demonstrate that the proposed approach improves both anomaly detection accuracy and reconstruction under sensor corruption. Overall, this framework bridges the gap between analytical robustness guarantees and practical fault-tolerant multimodal learning.

多模态学习容错机制异常检测自监督

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。