提出实时防护机制，防止大模型代理调用危险工具
ToolSafe: Enhancing Tool Invocation Safety of LLM-based agents via Proactive Step-level Guardrail and Feedback
- 通过多任务强化学习构建安全拦截模型，提前判断工具调用风险
- 在对抗攻击下将有害调用减少65%，任务完成率提升10%
- 适用于需高安全性的自动化代理系统，如金融或医疗场景
基于大模型的智能体可通过调用外部工具与环境交互，但其能力扩展也带来了安全风险。实时监控步骤级工具调用行为并提前干预至关重要，但当前研究仍不足。本文首先构建了TS-Bench——一个面向大模型智能体步骤级工具调用安全检测的新基准。随后提出基于多任务强化学习的防御模型TS-Guard，通过分析交互历史，主动识别执行前的不安全调用行为，评估请求危害性与动作-攻击关联性，生成可解释且泛化性强的安全判断与反馈。进一步提出TS-Flow框架，以防护-反馈驱动推理，使ReAct类智能体在对抗攻击下有害工具调用平均降低65%，良性任务完成率提升约10%。
原文摘要 · Abstract (English)
While LLM-based agents can interact with environments via invoking external tools, their expanded capabilities also amplify security risks. Monitoring step-level tool invocation behaviors in real time and proactively intervening before unsafe execution is critical for agent deployment, yet remains under-explored. In this work, we first construct TS-Bench, a novel benchmark for step-level tool invocation safety detection in LLM agents. We then develop a guardrail model, TS-Guard, using multi-task reinforcement learning. The model proactively detects unsafe tool invocation actions before execution by reasoning over the interaction history. It assesses request harmfulness and action-attack correlations, producing interpretable and generalizable safety judgments and feedback. Furthermore, we introduce TS-Flow, a guardrail-feedback-driven reasoning framework for LLM agents, which reduces harmful tool invocations of ReAct-style agents by 65 percent on average and improves benign task completion by approximately 10 percent under prompt injection attacks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。