检测大模型生成文本中的水印片段,识别被修改部分。
Segmenting Watermarked Texts From Language Models
- 用随机性检验与变化点检测定位水印区域。
- 在多种模型上准确识别水印子串,误报率低。
- 适合内容溯源、防伪与可信生成研究者使用。
水印技术通过在生成内容中嵌入几乎不可察觉的统计信号,以追踪来源。本文研究一个场景:不可信第三方用户向可信语言模型(LLM)提供提示,由提供商生成带水印的文本。若用户发布该文本,检测器可追溯其来源。用户可能通过替换、插入或删除修改文本。本文提出一种统计方法,从检测器视角判断文本是否由大模型生成,并进一步将文本分割为带水印和无水印子串。方法基于随机化检验与变化点检测,可控制第一类与第二类错误,在多个语言模型生成的文本上验证有效,结果令人鼓舞。代码已公开于 https://github.com/doccstat/llm-watermark-cpd。
原文摘要 · Abstract (English)
Watermarking is a technique that involves embedding nearly unnoticeable statistical signals within generated content to help trace its source. This work focuses on a scenario where an untrusted third-party user sends prompts to a trusted language model (LLM) provider, who then generates a text from their LLM with a watermark. This setup makes it possible for a detector to later identify the source of the text if the user publishes it. The user can modify the generated text by substitutions, insertions, or deletions. Our objective is to develop a statistical method to detect if a published text is LLM-generated from the perspective of a detector. We further propose a methodology to segment the published text into watermarked and non-watermarked sub-strings. The proposed approach is built upon randomization tests and change point detection techniques. We demonstrate that our method ensures Type I and Type II error control and can accurately identify watermarked sub-strings by finding the corresponding change point locations. To validate our technique, we apply it to texts generated by several language models with prompts extracted from Google's C4 dataset and obtain encouraging numerical results. We release all code publicly at https://github.com/doccstat/llm-watermark-cpd.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。