提出可量化模型安全拒绝能力的对数间隙指标,并用它发现高效防御攻击的短后缀。
Logit-Gap Steering: A Forward-Pass Diagnostic for Alignment Robustness
- 用首步解码时拒绝与肯定词的对数差衡量模型安全裕度。
- 97.5%-99.8%有毒提示下对数间隙扩大,且与真实拒绝率一致。
- 无需梯度、仅需前向传播,快速生成低困惑度有效防御后缀。
RLHF式对齐训练使语言模型拒绝不安全请求,但其拒绝机制有多少安全余量?我们提出拒绝-肯定对数间隙:首步解码时最高拒绝标记对数与最高肯定标记对数之差,该标量可量化每条提示的安全裕度。实证显示,对三类模型在97.5%-99.8%的毒害提示上对数间隙被扩大;中位数间隙缩小程度与真拒绝率排名高度相关(内部一致性检验)。为验证指标实用性,我们提出对数间隙引导法——一种无需梯度、仅依赖前向传播的方法,可发现长度小于10个词元的分布内后缀,其累积效应能关闭间隙。每类模型约需26,000次前向传播(单张A100上约2分钟),仅为一次GCG搜索的1/125。在0.5B–2B模型上发现的后缀可无修改迁移至72B模型。8个后缀集成在AdvBench和HarmBench上实现38%-96%真拒绝率,多数后缀困惑度比GCG均值低10^3–10^4倍。已发表的困惑度过滤防御导致GCG失效(64.7%→1.0%),而我们的后缀保持稳定(76.9%→76.0%)。结果表明当前对齐裕度虽持续存在但可能较薄,且防御策略必须考虑分布内后缀。
原文摘要 · Abstract (English)
RLHF-style alignment trains language models to refuse unsafe requests, but how much operational margin does this refusal rest on? We introduce the refusal-affirmation logit gap: the difference between the top refusal-token logit and the top affirmative-token logit at the first decoding step. This single scalar quantifies the per-prompt safety margin that alignment provides. Empirically, alignment widens the gap on 97.5-99.8% of toxic prompts across three model families, and median gap closure co-varies with True-ASR ranking across suffix strategies (an internal consistency check, since our method optimises gap closure). To validate the metric's practical significance, we present logit-gap steering, a gradient-free, forward-pass-only method that discovers short in-distribution suffixes ($<$10 tokens per component) whose cumulative effect closes the gap. The method requires ${\approx}26{,}000$ forward-pass equivalents per family (${\approx}2$~min on one A100), ${\approx}125\times$ less than a single GCG search. Suffixes discovered on 0.5B--2B models transfer without modification to 72B within family. An 8-suffix ensemble reaches 38-96\% True ASR across 13 models on AdvBench and HarmBench, with most suffixes having $10^{3}$-$10^{4}\times$ lower perplexity than GCG-meaning published perplexity-filter defenses that collapse GCG (64.7%$\to$1.0%) leave our suffixes nearly intact (76.9%$\to$76.0%). These results demonstrate that current alignment margins, while consistently present, can be thin and efficiently measurable, and that defense strategies must account for in-distribution suffixes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。