提出黑盒水印框架NSmark,抵御模型输出空间攻击。
NSmark: Null Space Based Black-box Watermarking Defense Framework for Language Models
- 基于输出矩阵零空间不变性设计水印方案。
- 在预训练与下游任务中验证水印鲁棒性与性能保持。
- 适合需保护模型知识产权的研究者使用。
语言模型已成为关键知识产权资产,亟需保护。尽管已有多种水印策略,但普遍易受线性功能等价攻击(LFEA)影响,该攻击无需知晓水印方案或训练数据即可破坏多数白盒水印。本文将LFEA攻击扩展至常见的黑盒场景,聚焦最后一层输出(称作LL-LFEA),发现输出矩阵的零空间在攻击下保持不变。基于此,提出NSmark——一种任务无关、可抵抗LL-LFEA攻击的黑盒水印框架。NSmark包含三个阶段:(i) 利用所有者数字签名生成水印,结合扩频调制提升鲁棒性;(ii) 通过输出映射提取器嵌入水印,在保持模型性能的同时最大化水印容量;(iii) 验证通过提取率与零空间一致性评估。在预训练及下游任务上的大量实验表明,该方法具备有效性、可扩展性、可靠性、保真度和鲁棒性。代码已开源:https://github.com/dongdongzhaoUP/NSmark。
原文摘要 · Abstract (English)
Language models (LMs) have emerged as critical intellectual property (IP) assets that necessitate protection. Although various watermarking strategies have been proposed, they remain vulnerable to Linear Functionality Equivalence Attack (LFEA), which can invalidate most existing white-box watermarks without prior knowledge of the watermarking scheme or training data. This paper analyzes and extends the attack scenarios of LFEA to the commonly employed black-box settings for LMs by considering Last-Layer outputs (dubbed LL-LFEA). We discover that the null space of the output matrix remains invariant against LL-LFEA attacks. Based on this finding, we propose NSmark, a black-box watermarking scheme that is task-agnostic and capable of resisting LL-LFEA attacks. NSmark consists of three phases: (i) watermark generation using the digital signature of the owner, enhanced by spread spectrum modulation for increased robustness; (ii) watermark embedding through an output mapping extractor that preserves the LM performance while maximizing watermark capacity; (iii) watermark verification, assessed by extraction rate and null space conformity. Extensive experiments on both pre-training and downstream tasks confirm the effectiveness, scalability, reliability, fidelity, and robustness of our approach. Code is available at https://github.com/dongdongzhaoUP/NSmark.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。