让扩散语言模型在任意生成顺序下都能实时识别并终止有害内容。
A2D: Any-Order, Any-Step Safety Alignment for Diffusion Language Models
- 通过随机掩码在词元层面直接对齐安全策略，实现细粒度控制。
- 将DIJA攻击成功率从80%以上降至1.3%以下，支持实时中断生成。
- 适合需要高安全性的生成式AI系统，如内容审核与对话机器人。
扩散大语言模型（dLLMs）支持任意顺序生成，但这种灵活性扩大了攻击面：有害内容可能出现在任意位置，且基于模板的预填充攻击（如DIJA）可绕过响应级拒绝机制。我们提出A2D（任意顺序、任意步骤防御），一种词元级对齐方法，当检测到有害内容时，使dLLMs发出[EOS]拒绝信号。通过在随机掩码下进行词元级安全对齐，A2D在多种条件下均有效抵御任意解码顺序和任意步骤预填充攻击。同时支持实时监控：模型可开始生成，但一旦发现不安全延续即自动终止。在安全基准测试中，A2D持续阻止有害输出，将DIJA成功率从超过80%降至接近零（LLaDA-8B-Instruct上为1.3%，Dream-v0-Instruct-7B上为0.0%），阈值化的[EOS]概率实现早期拒绝，最快可提升19.3倍的安全终止速度。
原文摘要 · Abstract (English)
Diffusion large language models (dLLMs) enable any-order generation, but this flexibility enlarges the attack surface: harmful spans may appear at arbitrary positions, and template-based prefilling attacks such as DIJA bypass response-level refusals. We introduce A2D (Any-Order, Any-Step Defense), a token-level alignment method that aligns dLLMs to emit an [EOS] refusal signal whenever harmful content arises. By aligning safety directly at the token-level under randomized masking, A2D achieves robustness to both any-decoding-order and any-step prefilling attacks under various conditions. It also enables real-time monitoring: dLLMs may begin a response but automatically terminate if unsafe continuation emerges. On safety benchmarks, A2D consistently prevents the generation of harmful outputs, slashing DIJA success rates from over 80% to near-zero (1.3% on LLaDA-8B-Instruct, 0.0% on Dream-v0-Instruct-7B), and thresholded [EOS] probabilities allow early rejection, yielding up to 19.3x faster safe termination.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。