将水印逻辑嵌入模型权重,实现难以移除的开源大模型内容溯源
OpenStamp: A Watermark for Open-Source Language Models

- 通过修改最终投影层将水印编码进模型权重
- 检测准确率更高,对模型性能影响极小
- 抗改写和后期微调攻击,适合开发者直接使用
随着大语言模型生成内容日益普及,水印技术被视为区分机器生成与人工撰写内容的有力手段。现有方法通过调整词元采样概率嵌入隐秘信号,但不适用于开源模型——用户可白盒访问并轻易禁用。本文提出OpenStamp,将水印逻辑直接编码于模型权重中,仅修改最终投影(或称去嵌入)层。在两个模型上的实验表明,OpenStamp在保持模型能力几乎不变的前提下,检测性能优于先前方法。植入的水印经设计和实证验证,对改写攻击更具鲁棒性,且更难通过后期微调去除。为便于开发者使用,我们发布了代码及4个主流开源模型的水印版本。
原文摘要 · Abstract (English)
With the growing prevalence of large language model (LLM) generated content, watermarking is considered a promising approach for attributing text to LLMs and distinguishing it from human-written content. A prominent class of techniques embeds subtle but detectable signals in generated text by modifying token sampling probabilities. However, such methods are unsuitable for open-source models, where users have white-box access and can easily disable watermarking during inference. In this work, we introduce OpenStamp, a watermarking technique that encodes the watermarking logic directly into the model weights by modifying only the final projection, or unembedding, layer. Through experiments across two models, we show that OpenStamp achieves superior detection performance, with minimal degradation in model capabilities compared to prior methods. The implanted watermark is explicitly designed, and empirically confirmed, to be more robust to paraphrasing attacks and harder to scrub off through post-hoc fine-tuning than prior open-source watermarks. To enable developers to watermark their models, we release our code alongside watermarked versions of 4 popular open-source models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。