arXiv:2505.11063cs.AIcs.CR2025-05中稿 · ICML被引 13

通过修正思维防止错误行为,让AI代理更安全可靠。

Think Twice Before You Act: Enhancing Agent Behavioral Safety with Thought Correction

  • 在思维阶段进行因果修正,不改动原模型即可提升安全性。
  • 在多个测试中将安全率从50%提升至平均90%,高出现有方法23%。
  • 轻量级设计适合各类AI代理,部署成本低,适合实际应用。

基于大模型的智能体通过迭代推理、工具调用和环境交互完成复杂任务,其中每个中间思维直接影响后续行动。思维中的微小偏差可能引发不可控的不安全行为,而现有防护机制通常仅作用于最终输出,或需侵入式修改模型。本文提出Thought-Aligner——一种轻量级、可插拔的安全模型,可在执行动作前对不安全思维进行因果修正,无需改变底层代理。修正后的思维反馈回代理,引导其决策与工具使用向更安全方向演进。该方法仅作用于思维层面,具备模型无关性,可适配多种智能体框架。我们在十个风险场景下,通过两阶段对比学习训练Thought-Aligner,利用成对生成的安全与不安全思维进行优化。在多个智能体安全基准和六种LLM上的实验表明,Thought-Aligner将行为安全率从约50%提升至平均90%,优于当前最优防护方案约23%,同时帮助性提升约5%。该方法每步延迟低,开销极小,支持可扩展部署。我们已公开发布Thought-Aligner-7B模型(https://huggingface.co/WhitzardAgent/Thought-Aligner-7B)。

原文摘要 · Abstract (English)

LLM-based agents solve complex tasks through iterative reasoning, tool use, and environment interaction, where each intermediate thought directly shapes subsequent actions. Small deviations in these thoughts can therefore propagate into unsafe behaviors, yet existing guardrails typically operate only on final outputs or require intrusive model modifications. We introduce Thought-Aligner, a lightweight plug-in safety model that performs causal correction on unsafe thoughts before action execution, without altering the underlying agent. The corrected thoughts are fed back into the agent, steering its decision process and tool use toward safer trajectories. Because it operates solely at the thought level, Thought-Aligner is model-agnostic and can be integrated into diverse agent frameworks. We train Thought-Aligner via two-stage contrastive learning on paired safe and unsafe thoughts generated across ten risk scenarios. Experiments on diverse agent-safety benchmarks and six LLMs show that Thought-Aligner increases behavioral safety from about 50% without protection to around 90% on average, exceeding state-of-the-art guardrails by roughly 23%, while also improving helpfulness by about 5%. The method incurs low per-step latency and minimal overhead, enabling scalable and practical deployment. We publicly release Thought-Aligner-7B at https://huggingface.co/WhitzardAgent/Thought-Aligner-7B.

AI安全思维修正轻量模型智能体

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。