arXiv:2501.13941cs.CRcs.AI2025-01ICML被引 14

给大模型加结构水印,生成无延迟且难移除。

GaussMark: A Practical Approach for Structural Watermarking of Language Models

  • 通过向模型权重添加高斯噪声实现结构化水印
  • 水印可被密钥持有者统计检测,且不影响生成速度与质量
  • 适合需要溯源的AI内容发布场景

大语言模型生成高质量文本的能力带来伦理与安全挑战,亟需可识别生成来源的技术。现有水印方法多在词元层面操作,存在延迟高、质量下降或鲁棒性差等问题。本文提出GaussMark,一种基于高斯独立性检验的结构水印方案,将水印嵌入模型权重本身。该方法利用微小高斯扰动不影响模型性能的特性,在不增加生成延迟的前提下,使提供商可通过密钥对输出进行统计检测。我们给出了形式化的有效性与检出力边界。大量实验表明,GaussMark在插入、删除、替换及跨语言翻译等干扰下仍保持可靠,且模型质量几乎无损失。

原文摘要 · Abstract (English)

Recent advances in Large Language Models (LLMs) have led to significant improvements in natural language processing tasks, but their ability to generate human-quality text raises significant ethical and operational concerns in settings where it is important to recognize whether or not a given text was generated by a human. Thus, recent work has focused on developing techniques for watermarking LLM-generated text, i.e., introducing an almost imperceptible signal that allows a provider equipped with a secret key to determine if given text was generated by their model. Current watermarking techniques are often not practical due to concerns with generation latency, detection time, degradation in text quality, or robustness. Many of these drawbacks come from the focus on token-level watermarking, which ignores the inherent structure of text. In this work, we introduce a new scheme, GaussMark, that is simple and efficient to implement, has formal statistical guarantees on its efficacy, comes at no cost in generation latency, and embeds the watermark into the weights of the model itself, providing a structural watermark. Our approach is based on Gaussian independence testing and is motivated by recent empirical observations that minor additive corruptions to LLM weights can result in models of identical (or even improved) quality. We show that by adding a small amount of Gaussian noise to the weights of a given LLM, we can watermark the model in a way that is statistically detectable by a provider who retains the secret key. We provide formal statistical bounds on the validity and power of our procedure. Through an extensive suite of experiments, we demonstrate that GaussMark is reliable, efficient, and relatively robust to corruptions such as insertions, deletions, substitutions, and roundtrip translations and can be instantiated with essentially no loss in model quality.

模型水印结构化高斯噪声

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。