arXiv:2506.07564cs.AIcs.CL2025-06被引 27

为大模型智能体设计安全协议,防止信息污染与冲突。

SAFEFLOW: A Principled Protocol for Trustworthy and Transactional Autonomous Agent Systems

  • 通过细粒度信息流控制追踪数据来源与安全标签
  • 支持多智能体并发执行与冲突自动解决,保持全局一致
  • 适合构建高可靠性、抗攻击的自主系统开发者使用

大语言模型和视觉语言模型的发展使智能体具备复杂推理与多模态工具调用能力。然而现有框架仍脆弱,缺乏安全信息流、可靠性和多智能体协调的机制。本文提出SAFEFLOW协议框架,实现细粒度信息流控制(IFC),精确追踪所有数据在智能体、工具、用户和环境间交换时的来源、完整性与机密性。通过约束大模型推理尊重安全标签,防止不可信输入污染高完整性决策。为保障并发多智能体环境下的鲁棒性,SAFEFLOW引入事务化执行、冲突消解与共享状态的安全调度,维持全局一致性。进一步设计了预写日志、回滚和安全缓存等机制,提升对运行时错误和策略违规的韧性。我们构建了SAFEFLOWBENCH基准套件,用于评估智能体在对抗性、噪声及并发条件下的可靠性。大量实验表明,基于SAFEFLOW的智能体在恶劣环境中仍保持出色任务性能与安全保证,显著优于当前最先进方法。SAFEFLOW与SAFEFLOWBENCH共同为可信赖、稳健、安全的智能体生态系统奠定基础。

原文摘要 · Abstract (English)

Recent advances in large language models (LLMs) and vision-language models (VLMs) have enabled powerful autonomous agents capable of complex reasoning and multi-modal tool use. Despite their growing capabilities, today's agent frameworks remain fragile, lacking principled mechanisms for secure information flow, reliability, and multi-agent coordination. In this work, we introduce SAFEFLOW, a new protocol-level framework for building trustworthy LLM/VLM-based agents. SAFEFLOW enforces fine-grained information flow control (IFC), precisely tracking provenance, integrity, and confidentiality of all the data exchanged between agents, tools, users, and environments. By constraining LLM reasoning to respect these security labels, SAFEFLOW prevents untrusted or adversarial inputs from contaminating high-integrity decisions. To ensure robustness in concurrent multi-agent settings, SAFEFLOW introduces transactional execution, conflict resolution, and secure scheduling over shared state, preserving global consistency across agents. We further introduce mechanisms, including write-ahead logging, rollback, and secure caches, that further enhance resilience against runtime errors and policy violations. To validate the performances, we built SAFEFLOWBENCH, a comprehensive benchmark suite designed to evaluate agent reliability under adversarial, noisy, and concurrent operational conditions. Extensive experiments demonstrate that agents built with SAFEFLOW maintain impressive task performance and security guarantees even in hostile environments, substantially outperforming state-of-the-art. Together, SAFEFLOW and SAFEFLOWBENCH lay the groundwork for principled, robust, and secure agent ecosystems, advancing the frontier of reliable autonomy.

智能体系统信息安全多智能体协议设计

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。