提出一种概率均衡的文本水印方法,兼顾可检测性与生成质量。
WaterMod: Modular Token-Rank Partitioning for Probability-Balanced LLM Watermarking
- 按概率排序词汇并分组,确保高概率词不被排除。
- 零比特时保留至少一个高概率词,多比特时每步嵌入一位信息。
- 适用于新闻、代码等生成任务,适合需溯源的AI内容场景。
大型语言模型已能以人类水平的流畅度撰写新闻、法律分析和软件代码。与此同时,欧盟人工智能法案要求每段合成文本都必须带有不可察觉且可机器验证的溯源标记。传统基于对数概率的水印方法通过在每个解码步骤中选择伪随机的绿色词汇并提升其对数概率来满足要求,但随机划分可能导致最高概率词被排除,从而损害流畅性。WaterMod 通过一种概率感知的模块化规则缓解此问题:先按模型概率降序排列词汇,再根据余数模 k 分组,使相邻且语义相近的词分布在不同类别中;对选定类别施加小幅度偏置。在零比特设置(k=2)下,熵自适应门控选择偶数或奇数类作为绿色列表;由于前两个最高概率词分属不同类别,该选择嵌入可检测信号的同时保证至少一个高概率词可用。在多比特情形(k>2)下,当前载荷数字 d 选择满足 rank mod k = d 的类别进行偏置,每步嵌入一位 base-k 数字,实现细粒度溯源。相同的模运算机制同时支持二进制归属与丰富载荷。实验表明,WaterMod 在零比特和多比特设置下均保持强水印检测性能与高质量生成,在自然语言生成、数学推理和代码合成等多种任务中表现稳健。代码与数据已公开于 https://github.com/Shinwoo-Park/WaterMod。
原文摘要 · Abstract (English)
Large language models now draft news, legal analyses, and software code with human-level fluency. At the same time, regulations such as the EU AI Act mandate that each synthetic passage carry an imperceptible, machine-verifiable mark for provenance. Conventional logit-based watermarks satisfy this requirement by selecting a pseudorandom green vocabulary at every decoding step and boosting its logits, yet the random split can exclude the highest-probability token and thus erode fluency. WaterMod mitigates this limitation through a probability-aware modular rule. The vocabulary is first sorted in descending model probability; the resulting ranks are then partitioned by the residue rank mod k, which distributes adjacent-and therefore semantically similar-tokens across different classes. A fixed bias of small magnitude is applied to one selected class. In the zero-bit setting (k=2), an entropy-adaptive gate selects either the even or the odd parity as the green list. Because the top two ranks fall into different parities, this choice embeds a detectable signal while guaranteeing that at least one high-probability token remains available for sampling. In the multi-bit regime (k>2), the current payload digit d selects the color class whose ranks satisfy rank mod k = d. Biasing the logits of that class embeds exactly one base-k digit per decoding step, thereby enabling fine-grained provenance tracing. The same modular arithmetic therefore supports both binary attribution and rich payloads. Experimental results demonstrate that WaterMod consistently attains strong watermark detection performance while maintaining generation quality in both zero-bit and multi-bit settings. This robustness holds across a range of tasks, including natural language generation, mathematical reasoning, and code synthesis. Our code and data are available at https://github.com/Shinwoo-Park/WaterMod.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。