提出双空间平滑方法,让大模型版权验证更抗干扰。
DSSmoothing: Toward Certified Dataset Ownership Verification for Pre-trained Language Models via Dual-Space Smoothing
- 在嵌入空间和排列空间同时加扰动,提升水印鲁棒性。
- 在多个数据集上验证,对自适应攻击仍保持稳定性能。
- 适合关注模型版权保护的研究者与开发者使用。
大规模网络数据集推动了预训练语言模型(PLMs)的快速发展,但未经授权的数据使用引发了严重的版权问题。现有数据集所有权验证(DOV)方法通常假设水印在推理过程中保持稳定,但在自然噪声和对抗性扰动下这一假设常失效。本文提出首个灰盒设置下的认证式数据集所有权验证方法——双空间平滑(DSSmoothing),通过在嵌入空间引入连续扰动以捕捉语义鲁棒性,在排列空间进行可控的词元重排序以捕捉序列鲁棒性。DSSmoothing分为两阶段:第一阶段在两个空间协同嵌入触发信号,生成范数约束且鲁棒的水印数据集;第二阶段在验证时对两个空间分别应用随机平滑,计算可疑模型的水印鲁棒性(WR),并与一组良性模型的主概率(PP)值进行统计比较。理论上,该方法在有界双空间扰动下提供可证明的鲁棒性保障,确保WR始终超过PP。大量实验在多个代表性网络数据集上验证了其稳定可靠的性能,并展现出对潜在自适应攻击的强鲁棒性。代码已开源。
原文摘要 · Abstract (English)
Large web-scale datasets have driven the rapid advancement of pre-trained language models (PLMs), but unauthorized data usage has raised serious copyright concerns. Existing dataset ownership verification (DOV) methods typically assume that watermarks remain stable during inference; however, this assumption often fails under natural noise and adversary-crafted perturbations. We propose the first certified dataset ownership verification method for PLMs under a gray-box setting (i.e., the defender can only query the suspicious model but is aware of its input representation module), based on dual-space smoothing (i.e., DSSmoothing). To address the challenges of text discreteness and semantic sensitivity, DSSmoothing introduces continuous perturbations in the embedding space to capture semantic robustness and applies controlled token reordering in the permutation space to capture sequential robustness. DSSmoothing consists of two stages: in the first stage, triggers are collaboratively embedded in both spaces to generate norm-constrained and robust watermarked datasets; in the second stage, randomized smoothing is applied in both spaces during verification to compute the watermark robustness (WR) of suspicious models and statistically compare it with the principal probability (PP) values of a set of benign models. Theoretically, DSSmoothing provides provable robustness guarantees for dataset ownership verification by ensuring that WR consistently exceeds PP under bounded dual-space perturbations. Extensive experiments on multiple representative web datasets demonstrate that DSSmoothing achieves stable and reliable verification performance and exhibits robustness against potential adaptive attacks. Our code is available at https://github.com/NcepuQiaoTing/DSSmoothing.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。