通过分步干预提升扩散语言模型生成安全性,不降质量。
Adaptive Steering and Remasking for Safe Generation in Diffusion Language Models

- 在去噪每一步检测有害语义,动态重掩码并自适应引导。
- 攻击绕过率降至0.64%,生成质量接近原始模型。
- 无需微调,可直接插入现有扩散模型使用。
扩散语言模型(DLMs)通过迭代去噪和双向优化生成文本,是自回归模型的有前景替代方案。然而,其迭代生成过程会引入独特安全漏洞:中间步骤产生的有害标记会在后续优化中传播,最终导致不安全输出。现有修复方法要么无法生成安全内容,要么虽安全但质量低下。为此,我们提出一种推理阶段防御框架,通过在去噪过程中分步干预,提升安全性而不牺牲输出质量。核心是对比安全方向(SGD),该方向捕捉有害与安全生成之间的语义边界。利用SGD评估每一步生成标记与有害语义的对齐程度;一旦检测到有害性,即重掩码对应标记,并以自适应强度重新启动去噪过程,强度根据危害程度动态调整。作为即插即用模块,本方法无需额外微调,可直接集成至现成的扩散模型。实验表明,该方法将攻击绕过率降至0.64%,同时保持生成质量接近原模型性能,验证了分步干预的有效性。代码已开源。
原文摘要 · Abstract (English)
Diffusion Language Models (DLMs) provide a promising alternative to autoregressive language models by generating text through iterative denoising and bidirectional refinement. However, this iterative generation paradigm also introduces unique safety vulnerabilities when harmful tokens generated at intermediate denoising steps propagate through subsequent refinement processes and eventually induce unsafe outputs. While there are a few attempts to remedy this issue, they either fail to generate safe outputs or generate safe yet low-quality outputs. This motivates us to propose an inference-time defense framework based on the step-wise intervention during the denoising process, which then improves the safety without compromising the output quality. The key component of our framework is a contrastive safety direction (SGD), a latent direction that captures the semantic boundary between harmful and safe generations. We leverage SGD to assess the alignment of generated tokens with harmful semantics at each denoising step. When harmful alignment is detected, our method remasks the corresponding tokens and resumes the denoising process with adaptive steering, where the steering strength is modulated according to the estimated degree of harmfulness. As a plug-and-play module, our method circumvents the need for additional fine-tuning and can be directly incorporated into off-the-shelf diffusion models. The experimental results show that our approaches reduce jailbreak success rates to 0.64% while preserving generation quality close to the original model performance. This confirms the effectiveness of step-wise intervention for safe diffusion language model generation. Our code is available at https://github.com/leeyejin1231/DLM_Steering_Remasking.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。