arXiv:2502.05213cs.CRcs.AI2025-02

让大模型生成文本自带可追踪的多比特水印,防抄袭更高效。

DERMARK: A Dynamic, Efficient and Robust Multi-bit Watermark for Large Language Models

  • 动态分段嵌入水印,按需调整每比特的文本长度。
  • 每比特所需令牌数减少25%,嵌入时间缩短50%。
  • 抗编辑和擦除攻击能力强,适合版权保护场景。

随着大语言模型(LLM)能力增强,其生成内容的版权侵权问题日益突出。为追踪未经授权的分发或转售,已有研究提出在文本中嵌入水印。现有方法多采用一比特水印,仅能判断文本是否由特定模型生成;而多比特水印可编码更丰富信息,识别具体模型与用户。然而,当前多比特方法未考虑文本的水印容量,尤其在低熵文本中易失败。本文分析发现水印嵌入服从正态分布,据此推导出最优分段公式。基于此,提出DERMARK:一种动态、高效且鲁棒的多比特水印方法,在推理时对每个水印比特动态划分变量长度的文本段。该方法无需生成额外中间矩阵,开销极小,并通过最小化提取损失提升对文本编辑的鲁棒性。实验表明,相比最先进方法,本方法平均减少25%的每比特令牌数,嵌入时间降低50%,且在文本修改和水印擦除攻击下仍保持高鲁棒性。

原文摘要 · Abstract (English)

As large language models (LLMs) grow more powerful, concerns over copyright infringement of LLM-generated texts have intensified. LLM watermarking has been proposed to trace unauthorized redistribution or resale of generated content by embedding identifiers within the text. Existing approaches primarily rely on one-bit watermarking, which only verifies whether a text was generated by a specific LLM. In contrast, multi-bit watermarking encodes richer information, enabling the identification of the specific LLM and user involved in generated or distributed content. However, current multi-bit methods directly embed the watermark into the text without considering its watermark capacity, which can result in failures, especially in low-entropy texts. In this paper, we analyze that the watermark embedding follows a normal distribution. We then derive a formal inequality to optimally segment the text for watermark embedding. Building upon this, we propose DERMARK, a dynamic, efficient, and robust multi-bit watermarking method that divides the text into variable-length segments for each watermark bit during the inference. Moreover, DERMARK incurs negligible overhead since no additional intermediate matrices are generated and achieves robustness against text editing by minimizing watermark extraction loss. Experiments demonstrate that, compared to SOTA, on average, our method reduces the number of tokens required per embedded bit by 25\%, reduces watermark embedding time by 50\%, and maintains high robustness against text modifications and watermark erasure attacks.

水印技术大模型版权保护多比特

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。