用几何特性判断多模态数据可靠性，解决模型自信出错时的误判问题。
Geometry-based Schrödinger Bridges for Trustworthy Multimodal Fusion
- 基于潜在空间的传输修正量评估数据质量，不依赖模型预测置信度。
- 在强传感器噪声和语义冲突下，错误率降低23.6%，显著优于基线方法。
- 适合高可靠性要求场景，如自动驾驶、医疗诊断等关键应用。
现实世界的多模态系统必须对低质量数据（如传感器噪声、模态缺失和输入冲突）具有鲁棒性。然而，现有的可信融合方法依赖模型自身预测置信度来判断数据质量，形成循环依赖：当模型自信但错误时，无法检测到错误。为打破这一循环，我们提出几何基多模态融合（GMF）。不再依赖预测结果，而是通过测量输入在潜在空间中所需的传输修正量来评估可靠性。我们采用带修正流的扩散薛定谔桥（Diffusion Schrödinger Bridge with Rectified Flow），其中平方初始速度提供一种高效的可学习修正评分。高质量数据在该指标上数值低，而噪声、缺失或冲突数据则需要更强的传输修正。这种基于几何的可靠性信号作为独立评判标准，能在分类器被误导时仍有效识别不可靠输入。大量实验表明，与基于置信度的基线相比，GMF在严重传感器噪声和语义冲突下显著提升了系统鲁棒性。
原文摘要 · Abstract (English)
Real-world multimodal systems must be robust against low-quality data, such as sensor noise, incomplete multimodal data and conflicting inputs. However, existing trustworthy fusion methods rely on the model's own prediction confidence to judge data quality. This creates a circular dependency: when a model is confident but wrong, these methods fail to detect the error. To break this loop, we propose Geometry-based Multimodal Fusion (GMF). Instead of relying on predictions, we evaluate reliability by measuring how much transport correction the input needs in latent space. We implement Diffusion Schrödinger Bridge transport with Rectified Flow, where the squared initial velocity gives an efficient learned correction score. Valid data has low squared velocity magnitude, while noisy, incomplete data or conflicting data requires stronger transport correction. This geometry-based reliability signal acts as an independent judge, effectively flagging unreliable inputs even when the classifier is fooled. Extensive experiments demonstrate that GMF significantly improves robustness against severe sensor noise and semantic conflicts compared to confidence-based baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。