arXiv:2608.09542cs.LGcs.AI2026-08

让大模型学会识别攻击机制,自动提升安全防御能力。

Dual-Adversarial Safety Alignment: Cultivating Intrinsic Threat Comprehension in LRMs

论文配图:Dual-Adversarial Safety Alignment: Cultivating Intrinsic Threat Comprehension in LRMs
图 1 · 摘自论文原文
  • 通过双对抗训练,让模型理解攻击背后的原理而非仅识别模式。
  • 仅用1000条合成数据,显著增强对越狱攻击的抵抗力。
  • 适合关注模型安全性与推理能力平衡的研究者使用。

大型推理模型(LRMs)在复杂任务中表现优异,但易受有害提示诱导产生不安全输出。现有方法多依赖直接拒绝或安全解释,聚焦于提示模式而非内在攻击机制,导致泛化能力差。本文提出AdvSafe框架,通过双对抗过程使模型内化不安全知识。第一阶段:自主代理动态生成欺骗性越狱提示,不断尝试突破强教师模型;第二阶段:被攻破的教师模型执行认知反制,揭示攻击成功原因及识别与缓解方法。该过程生成紧凑的推理数据集,学生模型通过学习获得内在威胁理解能力。实验表明,仅用1000条合成样本,AdvSafe对齐的模型在抵御越狱攻击上优于现有基线,且几乎无推理性能损失。同时,对分布外提示也展现更强鲁棒性,证明学习不安全知识可实现更优的鲁棒性-实用性权衡,并超越已见攻击模式。

原文摘要 · Abstract (English)

Large reasoning models (LRMs) achieve remarkable success on complex tasks but remain vulnerable to harmful prompts that induce unsafe outputs. Recent methods align LRMs using direct refusals or safety rationales, yet often focus on prompt patterns rather than intrinsic attack mechanisms. As a result, these pattern-centric alignments struggle to generalize across diverse jailbreaks, compromising adversarial robustness and reasoning utility. We propose AdvSafe, a dual-adversarial framework that enables LRMs to internalize unsafety knowledge by explicitly deconstructing adversarial mechanisms. This moves beyond pattern-dependent traces, fostering robust cognitive defense without compromising reasoning utility. Our pipeline operates via a two-phase adversarial game. First, in adversarial synthesis, an autonomous agent dynamically crafts deceptive jailbreak prompts, adapting its strategies to breach a strong teacher model. Second, in adversarial extraction, the breached teacher executes a cognitive counter-attack. For every successful jailbreak, the teacher unmasks the camouflage, explaining why the attack succeeds and how such prompts can be identified and mitigated. This dual-adversarial process yields a compact reasoning dataset capturing rich, generalizable unsafety knowledge. Student models trained on this dataset implicitly acquire safety alignment through intrinsic threat comprehension. Experiments show that with only 1K synthesized samples, AdvSafe-aligned LRMs achieve significantly stronger jailbreak robustness than existing baselines, with almost no utility degradation. Furthermore, AdvSafe improves robustness against out-of-distribution prompts, demonstrating that learning unsafety knowledge enables a superior robustness-utility trade-off and generalizes beyond seen attack patterns.

安全对齐对抗训练大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。