用逻辑推理守护智能体安全,防止恶意指令导致风险
ShieldAgent: Shielding Agents via Verifiable Safety Policy Reasoning

- 通过提取政策文档生成可验证的动作规则电路,实现安全策略推理
- 在3个基准上平均性能提升11.3%,召回率达90.1%,且减少64.7%的API调用
- 适用于需高安全性保障的智能体系统,如金融、隐私敏感场景
由基础模型驱动的自主智能体已在诸多真实场景中广泛应用,但极易受到恶意指令和攻击,可能导致隐私泄露与财务损失。现有大模型防护机制因智能体行为的复杂性与动态性而难以适用。为此,我们提出ShieldAgent,首个通过逻辑推理强制执行其他受保护智能体动作轨迹安全策略的防护智能体。ShieldAgent首先从政策文档中提取可验证规则,构建基于动作的概率规则电路;给定受保护智能体的动作轨迹,它检索相关规则电路并利用其工具库与可执行代码生成防护方案,实现形式化验证。此外,针对缺乏智能体防护基准的问题,我们构建了ShieldAgent-Bench数据集,包含3,000组安全相关的智能体指令与动作轨迹对,覆盖6个网页环境和7类风险,基于SOTA攻击生成。实验表明,ShieldAgent在ShieldAgent-Bench及三个现有基准上均达最优,平均性能超越先前方法11.3%,召回率高达90.1%;同时减少64.7%的API调用与58.2%的推理时间,展现高精度与高效性。
原文摘要 · Abstract (English)
Autonomous agents powered by foundation models have seen widespread adoption across various real-world applications. However, they remain highly vulnerable to malicious instructions and attacks, which can result in severe consequences such as privacy breaches and financial losses. More critically, existing guardrails for LLMs are not applicable due to the complex and dynamic nature of agents. To tackle these challenges, we propose ShieldAgent, the first guardrail agent designed to enforce explicit safety policy compliance for the action trajectory of other protected agents through logical reasoning. Specifically, ShieldAgent first constructs a safety policy model by extracting verifiable rules from policy documents and structuring them into a set of action-based probabilistic rule circuits. Given the action trajectory of the protected agent, ShieldAgent retrieves relevant rule circuits and generates a shielding plan, leveraging its comprehensive tool library and executable code for formal verification. In addition, given the lack of guardrail benchmarks for agents, we introduce ShieldAgent-Bench, a dataset with 3K safety-related pairs of agent instructions and action trajectories, collected via SOTA attacks across 6 web environments and 7 risk categories. Experiments show that ShieldAgent achieves SOTA on ShieldAgent-Bench and three existing benchmarks, outperforming prior methods by 11.3% on average with a high recall of 90.1%. Additionally, ShieldAgent reduces API queries by 64.7% and inference time by 58.2%, demonstrating its high precision and efficiency in safeguarding agents.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。