提出可实时审查工具调用的智能防护系统，提升大模型代理安全性。
StepGuard: Learning Step-Level Guardrails with Scalable Supervision and Safety-Utility Balancing
- 通过生成安全与危险轨迹对，实现步骤级行为监督训练。
- 在真实场景中将攻击成功率降低77.3%，仅损失2.8%任务效率。
- 适合需要高安全性的自动化代理应用，如智能客服或金融操作。
基于大模型的智能体通过调用外部工具与环境交互，但也带来文件篡改、信息泄露和未授权操作等安全风险。现有防护机制多在轨迹完成后评估，缺乏对每一步操作的预执行监控。本文提出StepGuard，一种可在执行前审查每一步工具调用的步骤级防护模型。为训练该模型，我们设计StepGen，一个自动数据生成引擎，能在相同上下文下生成具有不同风险操作的合法与非法轨迹。为进一步缓解过度防御与防御不足问题，提出Balance-GRPO算法，根据安全与危险动作的实际识别准确率动态调整学习权重。实验表明，StepGuard在开源权重防护模型中平均准确率最高，性能接近GPT-5.4。在AgentDojo和AgentDyn测试中，相比无防护设置，攻击成功率下降77.3%，而任务平均效用仅下降2.8个百分点。
原文摘要 · Abstract (English)
LLM-based agents can interact with external environments through tool invocation, but this capability also introduces security risks such as file modification, information leakage, and unauthorized actions. Existing guardrails often evaluate completed trajectories, leaving pre-execution monitoring of step-level actions underexplored. We propose StepGuard, a step-level guard model that can audit completed agent trajectories and check tool actions before they are executed. To train StepGuard, we introduce StepGen, an automatic data engine that generates safe and unsafe trajectories with the same context but different actions at the risky step. To further reduce over-defense and under-defense, we propose Balance-GRPO, which dynamically balances learning between safe and unsafe actions based on their observed accuracy. Experiments show that StepGuard achieves the highest average accuracy among open-weight guard models, with performance comparable to GPT-5.4. When used to guard agents on AgentDojo and AgentDyn, StepGuard reduces mean attack success rate by 77.3% relative to the no-guard setting, while mean utility drops by only 2.8 percentage points.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。