为非自回归的扩散语言模型设计了首个水印技术。
Watermarking Diffusion Language Models
- 在上下文不确定时仍基于期望值加水印
- 水印检测准确率超99%，且不影响生成质量
- 适用于需要可靠内容溯源的AI生成场景
我们提出了首个专为扩散语言模型（DLM）设计的水印方法。DLM是一种新兴的语言建模范式，可任意顺序生成词汇，与传统自回归语言模型（ARLM）的逐序列生成不同。现有ARLM水印方法依赖已生成的上下文，难以直接应用于DLM。本文通过：(i) 在部分上下文未确定时仍基于其期望值加水印；(ii) 优先选择能增强其他词水印强度的候选词作为上下文，实现有效水印。该方法无需修改原有检测器。实验表明，该水印在保持生成质量几乎不变的前提下，达到超过99%的真阳性率，并具备与现有ARLM水印相当的鲁棒性，首次实现了可靠的DLM水印。
原文摘要 · Abstract (English)
We introduce the first watermark tailored for diffusion language models (DLMs), an emergent LLM paradigm able to generate tokens in arbitrary order, in contrast to standard autoregressive language models (ARLMs) which generate tokens sequentially. While there has been much work in ARLM watermarking, a key challenge when attempting to apply these schemes directly to the DLM setting is that they rely on previously generated tokens, which are not always available with DLM generation. In this work we address this challenge by: (i) applying the watermark in expectation over the context even when some context tokens are yet to be determined, and (ii) promoting tokens which increase the watermark strength when used as context for other tokens. This is accomplished while keeping the watermark detector unchanged. Our experimental evaluation demonstrates that the DLM watermark leads to a >99% true positive rate with minimal quality impact and achieves similar robustness to existing ARLM watermarks, enabling for the first time reliable DLM watermarking.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。