新水印方法让大模型生成文本既难伪造又保持高质量
Majority Bit-Aware Watermarking For Large Language Models
- 通过多数位感知编码，放宽对优选词表大小的限制
- 在主流大模型上实现更高识别准确率和更优文本质量
- 适合需要防滥用且重视生成效果的AI系统
大型语言模型（LLMs）的广泛应用引发了其被用于生成有害或误导性内容的担忧。为应对这一问题，已有水印方法可在生成文本中嵌入可识别的多比特信息以追踪滥用行为。然而，现有方法常面临文本质量与解码准确率之间的根本权衡：为保证可检测的水印信号，编码时需限制优选词表（即绿色列表）规模，这不可避免地降低生成质量。为此，我们提出一种新的消息编码范式—— extit{多数位感知编码}，该策略不再依赖绿色列表大小来维持水印信号强度。这一设计使即使在使用大规模绿色列表的情况下，生成文本仍能保留强水印信号。我们实现了两种具体方案：MajorMark 和 MajorMark$^{+}$，后者特别针对长消息进行了优化。在最先进大模型上的大量实验表明，我们的方法相比先前基线，在解码准确率和文本质量方面均有显著提升。
原文摘要 · Abstract (English)
The growing deployment of Large Language Models (LLMs) has raised concerns about their misuse in generating harmful or deceptive content. To address this issue, watermarking methods have been proposed to embed identifiable multi-bit messages into generated text for misuse tracing. However, existing methods often suffer from a fundamental trade-off between text quality and decoding accuracy. In particular, they have to restrict the size of the preferred token set (i.e., green list) during encoding to maintain a detectable watermark signal for decoding, which inevitably degrades generation quality. To improve this trade-off, we propose a novel message encoding paradigm called \textit{majority bit-aware encoding}, which relaxes the watermark signal strength from the green list size. This strategy allows for a strong watermark signal to be preserved in generated texts even when using a large green list. We introduce two instantiations of this paradigm: MajorMark and MajorMark$^{+}$, where the latter is specifically optimized for long messages. Extensive experiments on state-of-the-art LLMs demonstrate that our methods achieve higher decoding accuracy and superior text quality compared to prior baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。