arXiv:2607.21600cs.AI2026-07中稿 · ICML

通过检测多模态一致性,高效识别恶意输入并降低攻击成功率。

Securing Multimodal AI through Internal Information Decomposition

论文配图:Securing Multimodal AI through Internal Information Decomposition
图 1 · 摘自论文原文
  • 监控多模态融合过程中的内部一致性,识别跨模态异常行为。
  • 在未见过的攻击下,将攻击成功率从90%以上降至15%以下。
  • 轻量级设计,延迟降低6倍,适用于实际部署的多模态系统。

多模态大语言模型引入了单模态系统不存在的攻击面:攻击者可将恶意意图分散到不同模态中以规避单模态防护。为此,我们提出以跨模态一致性作为检测信号,而非孤立分析各模态。核心观察是:良性输入促使文本与视觉独立推理产生兼容预测,融合后趋于稳定;而恶意干扰会破坏这种一致性,导致多模态行为异常。现有防御方法关注原始输入或输出,忽视内部融合过程,因而脆弱且计算开销大。本文提出FlowGuard,一种轻量级推理时框架,通过监测内部多模态一致性来检测有害输入。不同于依赖标量置信度的方法,FlowGuard借鉴部分信息分解思想,生成FlowVectors,量化跨模态冗余、协同效应及模态特异性主导程度,捕捉融合预测是否与单模态语义证据保持一致。在仅用良性数据训练的一类分类任务中,FlowGuard将未见攻击的攻击成功率从>90%降至<15%,实用损失低于3%,延迟最高降低6倍。结果表明,监控跨模态一致性为多模态推理提供了高效有效的防御手段。

原文摘要 · Abstract (English)

Multimodal large language models introduce attack surfaces absent in unimodal systems: adversaries can distribute malicious intent across modalities to evade unimodal safeguards. This motivates using cross-modal consistency as a detection signal rather than inspecting each modality in isolation. Our key observation is that benign inputs induce compatible predictive behavior from text-only and vision-only reasoning that stabilizes when fused, whereas adversarial manipulation disrupts this consistency, causing abnormal multimodal behavior. Existing defenses that examine raw inputs or outputs overlook this internal fusion process, rendering them brittle and computationally expensive. We propose FlowGuard, a lightweight inference-time framework that detects harmful inputs by monitoring internal multimodal consistency. Unlike approaches that rely on scalar confidence metrics, FlowGuard derives FlowVectors inspired by Partial Information Decomposition that quantify cross-modal redundancy, synergy, and modality-specific dominance, capturing whether fused multimodal predictions remain aligned with unimodal semantic evidence. In a one-class classification problem trained solely on benign data, FlowGuard reduces Attack Success Rates from >90% to <15% on unseen attacks, with <3% utility loss and up to a 6 times latency reduction. Our results demonstrate that monitoring cross-modal consistency offers an efficient and effective defense for multimodal reasoning.

多模态安全一致性检测推理防御轻量级

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。