改进大模型越狱后缀优化的候选选择策略,提升攻击成功率与稳定性。
TACS: Trajectory-Aware Candidate Selection for LLM Jailbreak Suffix Optimization

- 引入轨迹感知代理,避免仅看当前损失的短视选择。
- 在HarmBench上显著提升越狱成功率,搜索过程更稳定。
- 适合关注大模型安全与对抗攻击的研究者使用。
基于梯度的越狱后缀优化方法通常通过保留当前损失最低的候选来更新后缀。我们发现这种看似自然的设计本质上具有短视性:在当前步骤表现较好的候选,往往在后续搜索中无法产生更好的越狱效果,这体现为一种选择阶段的奖励欺骗。这表明,候选选择而非生成,是后缀优化中的隐藏瓶颈。为此,我们提出\ORMS{},一个面向越狱后缀优化的轨迹感知候选选择框架。该方法不仅依据即时损失评估候选,还引入轨迹感知代理,并通过参考策略正则化和判别器估计的卡方校正稳定选择过程,鼓励那些在后续步骤仍有效的选择。在HarmBench上的实验表明,\ORMS{}在相同搜索预算下持续优于强基线,显著提升了攻击成功率,同时在整个搜索过程中表现出更稳定的优化行为。研究结果强调,缓解由短视候选选择引发的选择阶段奖励欺骗,对提升越狱后缀优化至关重要。
原文摘要 · Abstract (English)
Gradient-based jailbreak suffix optimization methods typically update the suffix by retaining the candidate with the lowest current loss. We show that this seemingly natural design is fundamentally myopic: candidates that look better under the current-step proxy often fail to produce better jailbreak outcomes later in the search, revealing a form of selection-stage reward hacking. This suggests that candidate selection, rather than candidate generation alone, is a hidden bottleneck in suffix optimization. To address this issue, we propose TACS, a trajectory-aware candidate selection framework for jailbreak suffix optimization. Instead of selecting candidates solely by their immediate loss, TACS augments per-step evaluation with a trajectory-aware proxy and stabilizes selection with reference-policy regularization and a discriminator-estimated chi-squared correction, encouraging choices that remain effective beyond the current step. Experiments on HarmBench show that TACS consistently outperforms strong baselines under the same search budget, substantially improving attack success rates while exhibiting more stable optimization behavior throughout the search. Our findings highlight that mitigating selection-stage reward hacking caused by myopic candidate selection is critical for improving jailbreak suffix optimization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。