给闭源大模型的训练数据加可验证水印，防数据盗用。
Dataset Watermarking for Closed LLMs with Provable Detection
- 通过重写增加词对共现频率，嵌入数据级水印信号。
- 在仅1%混合数据下仍能以p<0.01检测水印。
- 水印不破坏模型性能，适合评测数据保护。
大型语言模型（LLMs）在大量松散收集的数据上进行预训练和微调，存在使用专有数据集或评估基准的风险。这催生了数据集水印需求：设计数据集，使模型训练后留下可检测的痕迹。以往工作主要针对开源模型。本文提出首个适用于闭源大模型、具备可证明检测能力的数据集水印方法。具体通过重写文本，提升随机选取词对的共现频率来嵌入数据级水印信号，并利用统计检验分析模型输出中的共现模式进行检测。我们在多个基础模型和基准数据集上评估，结果表明该方法在微调阶段可靠检测水印（p < 0.01）。值得注意的是，在水印数据仅占总微调令牌约1%的数据混合场景下，方法依然有效。此外，该方法保持了基准数据的可用性和语义完整性。
原文摘要 · Abstract (English)
Large language models (LLMs) are pre-trained and post-trained on vast amounts of loosely curated data, raising the possibility that these models may have been trained on proprietary datasets or the same benchmarks used for evaluation. This motivates the need for dataset watermarking: designing datasets such that training on them leaves detectable signatures in the resulting model. Prior work has explored this problem for open models. We introduce the first dataset watermarking method for closed LLMs with provable detection. In particular, we embed a dataset-level watermark signal by increasing the co-occurrence frequency of randomly selected word pairs through rephrasing, and detect it using a statistical test on co-occurrence patterns in model-generated outputs. We evaluate our method with multiple base models and benchmark datasets and show that it reliably detects the watermark ($p <0.01$) in the fine-tuning stage. Notably, our method remains effective in a data mixture setting where the watermarked dataset constitutes only approximately $1\%$ of the total fine-tuning tokens. Furthermore, we show that our method preserves the utility and semantic integrity of the benchmark.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。