为软提示设计可序列水印的版权审计方法,防伪且不影响模型性能。
SWAP: Towards Copyright Auditing of Soft Prompts via Sequential Watermarking
- 通过指定分布外类的顺序嵌入水印,避开主任务决策空间。
- 在11个数据集上验证有效,误报率低且对攻击有鲁棒性。
- 适合需要保护视觉语言模型软提示版权的研究者与开发者。
大规模视觉-语言模型(如CLIP)在多种下游任务中表现出色。软提示作为高效适配模型的模块,亟需有效的版权保护机制。本文研究如何审计可疑第三方模型是否包含受保护的软提示。现有技术无效:非侵入式方法因共享数据分布导致误报;侵入式方法在CLIP上无法植入功能触发器,传统DNN后门技术也因有害性和歧义性不适用。根本原因在于水印与主任务在相同决策空间但目标相反。为此,我们提出序列水印方法SWAP,将水印编码于防御者指定的分布外类别顺序中,利用CLIP的零样本预测能力。该水印位于更复杂空间,不改变原预测标签,减少与主任务冲突。我们设计了基于假设检验的验证协议,并提供理论分析。11个数据集上的实验表明SWAP有效、无害且抗攻击。
原文摘要 · Abstract (English)
Large-scale vision-language models, especially CLIP, have demonstrated remarkable performance across diverse downstream tasks. Soft prompts, as carefully crafted modules that efficiently adapt vision-language models to specific tasks, necessitate effective copyright protection. In this paper, we investigate model copyright protection by auditing whether suspicious third-party models incorporate protected soft prompts. While this can be viewed as a special case of model ownership auditing, our analysis shows that existing techniques are ineffective due to prompt learning's unique characteristics. Non-intrusive auditing is inherently prone to false positives when independent models share similar data distributions with victim models. Intrusive approaches also fail: backdoor methods designed for CLIP cannot embed functional triggers, while extending traditional DNN backdoor techniques to prompt learning suffers from harmfulness and ambiguity challenges. We find that these failures in intrusive auditing stem from the same fundamental reason: watermarking operates within the same decision space as the primary task yet pursues opposing objectives. Motivated by these findings, we propose sequential watermarking for soft prompts (SWAP), which implants watermarks into a different and more complex space. SWAP encodes watermarks through a specific order of defender-specified out-of-distribution classes, inspired by the zero-shot prediction capability of CLIP. This watermark, which is embedded in a more complex space, keeps the original prediction label unchanged, making it less opposed to the primary task. We further design a hypothesis-test-guided verification protocol for SWAP and provide a theoretical analysis of when verification works. Extensive experiments on 11 datasets demonstrate SWAP's effectiveness, harmlessness, and robustness against potential attacks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。