首个评估多轮工具使用智能体安全性的基准，发现风险显著上升。
Unsafer in Many Turns: Benchmarking and Defending Multi-Turn Safety Risks in Tool-Using Agents
- 构建多轮攻击序列，系统化测试工具使用中的安全漏洞。
- 多轮交互下攻击成功率平均提升16%，安全风险加剧。
- 提出无需训练的自探索防御机制，平均降低30%攻击成功率。
基于大语言模型的智能体能力日益增强，但其安全性滞后，导致能做与应做之间的差距扩大。这一差距在多轮交互和多种工具使用中进一步放大，现有基准未能覆盖此类风险。为此，我们提出一种系统性分类法，将单轮有害任务转化为多轮攻击序列，并构建首个用于评估多轮工具使用智能体安全性的基准MT-AgentRisk。实验显示，多轮设置下开放与闭源模型的攻击成功率（ASR）平均提升16%。为缩小此差距，我们提出ToolShield——一种无需训练、不依赖特定工具的自探索防御方法：当遇到新工具时，智能体自主生成测试用例，执行并观察下游影响，提炼安全经验用于部署。实验表明，该方法在多轮交互中平均降低ASR 30%。代码已开源：https://github.com/CHATS-lab/ToolShield。
原文摘要 · Abstract (English)
LLM-based agents are becoming increasingly capable, yet their safety lags behind. This creates a gap between what agents can do and should do. This gap widens as agents engage in multi-turn interactions and employ diverse tools, introducing new risks overlooked by existing benchmarks. To systematically scale safety testing into multi-turn, tool-realistic settings, we propose a principled taxonomy that transforms single-turn harmful tasks into multi-turn attack sequences. Using this taxonomy, we construct MT-AgentRisk (Multi-Turn Agent Risk Benchmark), the first benchmark to evaluate multi-turn tool-using agent safety. Our experiments reveal substantial safety degradation: the Attack Success Rate (ASR) increases by 16% on average across open and closed models in multi-turn settings. To close this gap, we propose ToolShield, a training-free, tool-agnostic, self-exploration defense: when encountering a new tool, the agent autonomously generates test cases, executes them to observe downstream effects, and distills safety experiences for deployment. Experiments show that ToolShield effectively reduces ASR by 30% on average in multi-turn interactions. Our code is available at https://github.com/CHATS-lab/ToolShield.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。