为无序生成语言模型设计新型水印，提升检测效率与鲁棒性
A Watermark for Order-Agnostic Language Models
- 基于马尔可夫链生成高频模式水印序列
- 在ProteinMPNN等模型上实现更高检测率与抗篡改能力
- 适合需要内容溯源的非顺序生成模型应用
统计水印技术在顺序生成的语言模型中已成熟，但无法直接用于无序生成的语言模型（order-agnostic LMs），因其令牌不按顺序生成。本文提出Pattern-mark，一种专为无序生成语言模型设计的基于模式的水印框架。我们开发了一种基于马尔可夫链的水印生成器，可生成具有高频关键模式的水印密钥序列；同时提出一种基于统计模式的检测算法，在检测阶段恢复密钥序列，并基于高频模式出现次数进行统计检验。在ProteinMPNN和CMLM等无序生成语言模型上的广泛评估表明，Pattern-mark在检测效率、生成质量与鲁棒性方面均表现更优，是无序生成语言模型的更优水印方案。
原文摘要 · Abstract (English)
Statistical watermarking techniques are well-established for sequentially decoded language models (LMs). However, these techniques cannot be directly applied to order-agnostic LMs, as the tokens in order-agnostic LMs are not generated sequentially. In this work, we introduce Pattern-mark, a pattern-based watermarking framework specifically designed for order-agnostic LMs. We develop a Markov-chain-based watermark generator that produces watermark key sequences with high-frequency key patterns. Correspondingly, we propose a statistical pattern-based detection algorithm that recovers the key sequence during detection and conducts statistical tests based on the count of high-frequency patterns. Our extensive evaluations on order-agnostic LMs, such as ProteinMPNN and CMLM, demonstrate Pattern-mark's enhanced detection efficiency, generation quality, and robustness, positioning it as a superior watermarking technique for order-agnostic LMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。