arXiv:2602.16977cs.LGcs.CR2026-02被引 2

提出抗失效对齐机制,让大模型拒绝违规请求更可靠。

Fail-Closed Alignment for Large Language Models

  • 通过迭代消除旧拒答方向,强制模型学习多条独立安全路径。
  • 在四种攻击下表现最优,且减少过度拒绝,保持生成质量。
  • 适合关注大模型安全性的研究人员和工程团队。

我们发现当前大语言模型对齐存在结构缺陷:现有拒绝机制为‘失效开放’型。尽管拒绝行为由多个隐层特征编码,但仅通过提示词劫持抑制单一主导特征,即可导致对齐崩溃,引发不安全生成。为此,我们提出‘失效封闭’对齐作为鲁棒安全设计原则:即使部分路径失效,拒绝机制仍应有效,需通过冗余且独立的因果路径实现。我们提出了具体实现方案——渐进式对齐框架,通过迭代识别并擦除已学拒绝方向,迫使模型在新的独立子空间中重建安全机制。在四种提示劫持攻击下,该方法展现出最强鲁棒性,同时缓解过度拒绝问题,维持生成质量,计算开销极小。机制分析证实,经该方法训练的模型确实在多个因果独立的拒绝方向上编码安全行为,无法被单一提示攻击同时抑制,为‘失效封闭’对齐提供了实证支持。

原文摘要 · Abstract (English)

We identify a structural weakness in current large language model (LLM) alignment: modern refusal mechanisms are fail-open. While existing approaches encode refusal behaviors across multiple latent features, suppressing a single dominant feature$-$via prompt-based jailbreaks$-$can cause alignment to collapse, leading to unsafe generation. Motivated by this, we propose fail-closed alignment as a design principle for robust LLM safety: refusal mechanisms should remain effective even under partial failures via redundant, independent causal pathways. We present a concrete instantiation of this principle: a progressive alignment framework that iteratively identifies and ablates previously learned refusal directions, forcing the model to reconstruct safety along new, independent subspaces. Across four jailbreak attacks, we achieve the strongest overall robustness while mitigating over-refusal and preserving generation quality, with small computational overhead. Our mechanistic analyses confirm that models trained with our method encode multiple, causally independent refusal directions that prompt-based jailbreaks cannot suppress simultaneously, providing empirical support for fail-closed alignment as a principled foundation for robust LLM safety.

大模型安全对齐机制鲁棒性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。