让智能体在实践中自我进化,同时优化安全策略与执行行为。
SafeEvolve: Harness-Policy Co-Evolution from Agent Experience for Safety Alignment

- 通过经验驱动的协同演化框架,同步优化安全控制与智能体策略。
- 在AgentDojo上实现3倍安全响应率降低,且良性任务完成率提升至61.86%。
- 适合关注智能体安全性与自主性平衡的研究者与开发者。
基于大模型的智能体性能由基础模型与交互时所用的控制机制共同决定,这使其面临最终输出有害内容及多步执行轨迹中累积风险的问题。现有安全对齐方法通常依赖外部控制机制更新或策略优化,但单独使用任一方式均无法实现运行时控制与内在安全的统一。本文提出SafeEvolve,一种基于智能体经验的自演化安全对齐框架。该框架利用已完成的策略轨迹中的安全经验,推动控制机制与策略的持续协同演化。在控制机制侧,将轨迹级安全证据转化为可审计、可逆的组件级更新,涵盖安全提示与分层技能;在策略侧,采用两阶段SFT-RL范式:先通过控制机制使用微调(SFT)引导策略主动利用演进后的控制机制,再通过验证器分解奖励的强化学习(RL)进一步塑造多步探索中的自主安全行为。实验表明,在智能体安全基准测试中,SafeEvolve相比现有基线实现了更优的安全-效用权衡。对于Qwen3.5-4B模型,其在AgentDojo上实现3倍的攻击成功率(ASR)降低,同时良性任务完成率从59.79%提升至61.86%。
原文摘要 · Abstract (English)
The performance of LLM-based agents is jointly shaped by the base model and the harness used when interacting with the environment. This exposes them to safety risks in both harmful final responses and multi-step execution trajectories. Existing safety alignment mechanisms often rely on either external harness updates or policy optimization, yet applying either paradigm in isolation fails to bridge runtime control with intrinsic safety. We propose SafeEvolve, an experience-driven self-evolving framework for agent safety alignment. SafeEvolve leverages safety experience from completed on-policy trajectories to drive a continual loop of harness-policy co-evolution. On the harness side, SafeEvolve converts trajectory-level safety evidence into bounded, component-level updates across safety prompt and hierarchical skills, yielding auditable and reversible harness artifacts. On the policy side, SafeEvolve follows a two-stage SFT-RL paradigm, where harness-use SFT bootstraps the policy to actively leverage evolved harness artifacts, and harness-augmented RL further shapes autonomous safety behaviors during multi-step exploration via verifier-decomposed rewards. Through harness-policy co-evolution, SafeEvolve converts safety experience into an evolved runtime harness and improved policy behavior. Experiments on agentic safety benchmarks show that SafeEvolve achieves a stronger safety-utility tradeoff than existing baselines. For Qwen3.5-4B, SafeEvolve achieves a $3\times$ ASR reduction on AgentDojo while improving benign utility from 59.79% to 61.86%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。