提出最优水印耦合策略，提升大模型文本水印检测力且控制质量损失。
Optimized Couplings for Watermarking Large Language Models
- 设计最优耦合机制，结合侧信息与词表随机划分
- 理论推导出在最坏分布下的检测率与最大代价
- 适用于需强检测力的可信生成场景
大型语言模型（LLMs）生成的文本在许多情况下已难以与人类写作区分。这推动了水印技术的发展，即以最小扰动在输出文本中嵌入‘信号’。本文分析了一次性设置下的文本水印问题。基于带侧信息的假设检验，我们建立了水印检测能力与生成文本质量畸变之间的基本权衡。我们认为，水印设计的关键在于构建共享侧信息与语言模型词汇表随机划分之间的耦合。我们的分析确定了在满足最小熵约束的最坏情况语言模型下一个字分布下最优的耦合与随机化策略。我们给出了所提方案下的闭式检测率表达式，并以最大最小方式量化了代价。最后，我们在合成数据和真实大模型水印任务中提供了多组数值结果，对比了所提方案与理论最优及现有方法的表现。代码已开源：https://github.com/Carol-Long/CC_Watermark。
原文摘要 · Abstract (English)
Large-language models (LLMs) are now able to produce text that is, in many cases, seemingly indistinguishable from human-generated content. This has fueled the development of watermarks that imprint a ``signal'' in LLM-generated text with minimal perturbation of an LLM's output. This paper provides an analysis of text watermarking in a one-shot setting. Through the lens of hypothesis testing with side information, we formulate and analyze the fundamental trade-off between watermark detection power and distortion in generated textual quality. We argue that a key component in watermark design is generating a coupling between the side information shared with the watermark detector and a random partition of the LLM vocabulary. Our analysis identifies the optimal coupling and randomization strategy under the worst-case LLM next-token distribution that satisfies a min-entropy constraint. We provide a closed-form expression of the resulting detection rate under the proposed scheme and quantify the cost in a max-min sense. Finally, we provide an array of numerical results, comparing the proposed scheme with the theoretical optimum and existing schemes, in both synthetic data and LLM watermarking. Our code is available at https://github.com/Carol-Long/CC_Watermark
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。