测试大模型代理在6个领域的580种场景下的安全表现，发现工具调用错误普遍存在。
GuardianAgentBench: Where Agents Fail and How to Guard Them
- 构建多阶段验证的基准测试GABench，覆盖6大领域580个场景
- 最强模型准确率仅74.8%，长序列规划是主要瓶颈
- 运行时结构干预可恢复19.9%失败案例，误报率仅0.5%
随着大语言模型代理越来越多地自主运行并访问工具与外部环境，确保其安全可靠行为变得至关重要。我们提出GuardianAgentBench（GABench），一个涵盖六个领域共580个场景的基准测试，评估了三个生产就绪框架：LangChain、LlamaIndex和Vectara。该基准包含严格的多阶段验证和五种对抗性攻击模式。六种最先进模型的实验表明，即使最强配置的整体准确率也仅为74.8%，并揭示了两种不同的失效模式：强模型会漏调所需工具，弱模型则误选或过度调用工具。性能随工具集规模和序列轮次深度单调下降，长周期规划成为更严峻的瓶颈。我们的防护机制在所有模型上均优于基于系统提示的防御方案，以0.5%的误报率恢复了19.9%的失败案例。结果表明，执行时的结构化干预可在不干扰正确行为的前提下提升安全性。
原文摘要 · Abstract (English)
As large language model agents increasingly operate autonomously with access to tools and external environments, ensuring their safe and reliable behavior becomes critical. We present GuardianAgentBench (GABench), a benchmark of 580 scenarios across six domains evaluated on three production-ready frameworks: LangChain, LlamaIndex, and Vectara. The benchmark incorporates rigorous multi-stage validation and five adversarial attack modes. Experiments with six state-of-the-art models reveal that even the strongest configuration achieves only 74.8% overall accuracy and expose two distinct failure regimes: stronger models under-call required tools, while weaker models mis-select and over-call tools. Performance degrades monotonically with both tool-set size and sequential turn depth, with long-horizon planning proving the steeper bottleneck. Our guardrail implementation consistently outperforms system-prompt-based defenses across all models, recovering 19.9% of failures at a false positive rate of just 0.5%. These results demonstrate that execution-time structural intervention improves safety without disrupting correct agent behavior.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。