通过修改模型内部参数实现水印嵌入,兼顾隐蔽性与高效提取
Yet Another Watermark for Large Language Models
- 直接操作LLM内部参数嵌入水印,而非仅调整采样或后处理
- 黑盒场景下仍可高效提取水印,且生成文本语义质量更高
- 适用于实际部署,为生成内容溯源提供新思路
现有大语言模型水印方法主要通过调整词元采样或后期处理来嵌入水印,缺乏与模型的内在耦合,可能显著降低生成文本的语义质量。基于训练或微调的传统方法虽可扩展至大模型,但多数局限于白盒场景,或因模型参数量庞大而计算开销巨大。本文提出一种新型水印框架,通过操纵大语言模型的内部参数嵌入水印,并可在不访问模型的情况下从生成文本中提取。相比现有方法,该方案将水印与模型内在参数紧密耦合,更好平衡了水印的鲁棒性与隐蔽性。此外,该方法支持在黑盒场景下高效提取,计算效率高。实验验证了其可行性、优越性与实用性。本工作提供了不同于主流研究的新视角,或可启发未来探索。
原文摘要 · Abstract (English)
Existing watermarking methods for large language models (LLMs) mainly embed watermark by adjusting the token sampling prediction or post-processing, lacking intrinsic coupling with LLMs, which may significantly reduce the semantic quality of the generated marked texts. Traditional watermarking methods based on training or fine-tuning may be extendable to LLMs. However, most of them are limited to the white-box scenario, or very time-consuming due to the massive parameters of LLMs. In this paper, we present a new watermarking framework for LLMs, where the watermark is embedded into the LLM by manipulating the internal parameters of the LLM, and can be extracted from the generated text without accessing the LLM. Comparing with related methods, the proposed method entangles the watermark with the intrinsic parameters of the LLM, which better balances the robustness and imperceptibility of the watermark. Moreover, the proposed method enables us to extract the watermark under the black-box scenario, which is computationally efficient for use. Experimental results have also verified the feasibility, superiority and practicality. This work provides a new perspective different from mainstream works, which may shed light on future research.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。