arXiv:2605.01133cs.CRcs.LG2026-05被引 2

现有防御方法在大模型多智能体系统中易被绕过,新方案利用置信度信号提升安全性。

When Embedding-Based Defenses Fail: Rethinking Safety in LLM-Based Multi-Agent Systems

论文配图:When Embedding-Based Defenses Fail: Rethinking Safety in LLM-Based Multi-Agent Systems
图 1 · 摘自论文原文
  • 通过分析嵌入相似性失效机制,发现攻击者可伪造与正常消息相近的嵌入
  • 引入消息置信度进行裁剪或降权,显著提升系统抗攻击能力
  • 适合关注多智能体系统安全、对抗攻击的研究者和开发者

基于大语言模型的多智能体系统(MAS)通过智能体间通信协作完成复杂任务,但通信环节也成为恶意攻击的入口,导致信息误导和群体决策偏差。现有基于嵌入的防御依赖恶意与良性消息在嵌入空间中的清晰分离,但攻击者可通过慢速漂移、良性包装、混沌注入等三类攻击,使恶意消息嵌入接近正常样本,从而绕过检测。我们从理论上分析此失效机理,并实证验证。研究进一步揭示:仅依赖嵌入的防御存在根本缺陷——忽视了在嵌入不可区分时仍具信息量的词元级置信度(如logits)。为此,我们提出在通信过程中使用置信度评分对消息进行裁剪或降权。实验表明,该方法在多种模型、数据集和通信拓扑下均提升了鲁棒性。此外,我们发现置信度信号随通信轮次衰减,凸显早期干预的重要性。这些发现为未来多智能体系统的攻防研究提供关键启示。

原文摘要 · Abstract (English)

Large language model (LLM)-powered multi-agent systems (MAS) enable agents to communicate and share information, achieving strong performance on complex tasks. However, this communication also creates an attack surface where malicious agents can propagate misinformation and manipulate group decisions, undermining MAS safety. Existing embedding-based defenses aim to detect and prune suspicious agents, but their effectiveness depends on a clear separation between the text embeddings of malicious and benign messages. Attackers can circumvent such defenses by crafting messages whose embeddings lie close to benign ones. We analyze this failure mode theoretically and validate it empirically with three attacks, Slow Drift, Benign Wrapper, and Chaos Seeding. Our analysis further reveals a fundamental limitation of embedding-based defenses: because they rely solely on the text embeddings, they ignore token-level confidence signals such as logits, which can remain informative when embeddings are not distinguishable under attack. We propose using confidence scores to prune or down-weight messages during MAS communication. Experiments show improved robustness across models, datasets, and communication topologies. Moreover, we find that the effectiveness of confidence signals decays over communication rounds, highlighting the importance of early intervention. This insights can inform and inspire future work on MAS attacks and defenses.

多智能体安全防御置信度大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。