用概率自动机实现语言模型水印，兼顾高效生成与强抗检测能力。
Watermarks for Language Models via Probabilistic Automata
- 基于概率自动机设计水印机制，提升生成多样性与计算效率。
- 实验显示在LLaMA-3B和Mistral-7B上兼具高鲁棒性与低检测开销。
- 提供可证明的不可检测性，适合对安全要求高的应用场景。
近期的语言模型水印方案实现了无失真嵌入和对编辑距离攻击的鲁棒性，但存在生成多样性受限和检测开销高的问题。与此同时，不可检测性（undetectability）成为研究热点，旨在使水印难以被敌手发现或伪造。本文提出一类基于概率自动机的新水印方案，包含两种实例：(i) 实用型方案具备指数级生成多样性和计算高效性；(ii) 理论型构造在密码学假设下具有形式化的不可检测性保证。在LLaMA-3B和Mistral-7B上的大量实验验证了该方案在鲁棒性与效率方面的优越表现。
原文摘要 · Abstract (English)
A recent watermarking scheme for language models achieves distortion-free embedding and robustness to edit-distance attacks. However, it suffers from limited generation diversity and high detection overhead. In parallel, recent research has focused on undetectability, a property ensuring that watermarks remain difficult for adversaries to detect and spoof. In this work, we introduce a new class of watermarking schemes constructed through probabilistic automata. We present two instantiations: (i) a practical scheme with exponential generation diversity and computational efficiency, and (ii) a theoretical construction with formal undetectability guarantees under cryptographic assumptions. Extensive experiments on LLaMA-3B and Mistral-7B validate the superior performance of our scheme in terms of robustness and efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。