用水印技术让模型训练数据归属检测更可行
Watermarking for Proprietary Dataset Protection
- 在部分水印数据上训练模型，利用残留水印信号推断数据归属
- 当数据子集暴露度高时，水印方法与传统损失法效果相当
- 适合关注数据隐私保护的研究者和模型开发者
现有研究认为，在现代语言建模中，训练数据成员身份推断是根本性难题。本文认为输出水印技术可使生成模型的训练成员身份检测更具可行性，基于先前发现语言模型在部分水印训练数据下仍存在残留水印‘放射性’。我们对比了基于水印的数据归属推断方法与传统的损失基成员身份推断方法，在不同假设条件下证明：当子集暴露程度足够高时，水印方法能达到相近的成员检测性能。
原文摘要 · Abstract (English)
A growing body of literature suggests that training data membership inference problems are fundamentally hard tasks in modern language modeling settings. We argue that output watermarking techniques are the right gadget to make training membership tests for generative models more tractable, based on prior results showing that language models exhibit residual watermark "radioactivity" under partially watermarked training datasets. We pit a watermark-based dataset inference approach head-to-head against traditional loss-based membership inference methods and show that watermarking can achieve comparable membership detection performance when subset exposure is high enough, under an alternate set of assumptions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。