提出新防御机制ControlValve，防止多智能体系统被控制流劫持。
Breaking and Fixing Defenses Against Control-Flow Hijacking in Multi-Agent Systems
- 基于控制流完整性与最小权限原则，生成合法执行路径图。
- 即使高级大模型也无法规避攻击，验证了现有防御缺陷。
- 适合关注AI系统安全、多智能体架构的开发者与研究者。
控制流劫持攻击通过操纵多智能体系统的协调机制，使系统执行不安全操作并泄露敏感信息。近期提出的防御措施（如LlamaFirewall）依赖于对智能体间通信的对齐检查，确保所有调用都与原始目标相关且可能进一步推进该目标。我们首先展示了即便使用先进大模型进行对齐检查，仍可绕过这些防御的控制流劫持攻击。我们指出，多智能体系统的安全性与功能性目标存在根本冲突，这一矛盾因“对齐”定义脆弱及检查器对执行上下文可见性不足而加剧。随后，我们提出、实现并评估了ControlValve——一种受控制流完整性与最小权限原则启发的新防御机制。ControlValve (1) 为多智能体系统生成允许的控制流图，(2) 强制所有执行遵循这些图，并结合零样本生成的上下文规则，对每个智能体调用进行约束。
原文摘要 · Abstract (English)
Control-flow hijacking attacks manipulate orchestration mechanisms in multi-agent systems into performing unsafe actions that compromise the system and exfiltrate sensitive information. Recently proposed defenses, such as LlamaFirewall, rely on alignment checks of inter-agent communications to ensure that all agent invocations are "related to" and "likely to further" the original objective. We start by demonstrating control-flow hijacking attacks that evade these defenses even if alignment checks are performed by advanced LLMs. We argue that the safety and functionality objectives of multi-agent systems fundamentally conflict with each other. This conflict is exacerbated by the brittle definitions of "alignment" and the checkers' incomplete visibility into the execution context. We then propose, implement, and evaluate ControlValve, a new defense inspired by the principles of control-flow integrity and least privilege. ControlValve (1) generates permitted control-flow graphs for multi-agent systems, and (2) enforces that all executions comply with these graphs, along with contextual rules (generated in a zero-shot manner) for each agent invocation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。