用对抗性双注入技术,让多模态模型自带版权追踪标记。
Echoes of ownership: Adversarial-guided dual injection for copyright protection in MLLMs
- 将图像作为可学习张量,通过双注入嵌入版权信息。
- 在微调后模型中仅触发版权响应,其他模型无反应。
- 适合关注模型版权保护的研究者与开发者。
随着多模态大语言模型(MLLMs)的快速部署,模型所有权争议日益频繁,知识产权保护面临严峻挑战。本文提出一种生成模型版权触发器的框架,使模型发布者能将可验证的所有权信息嵌入模型中。目标是构建仅在微调衍生模型中触发版权相关文本响应的触发图像,而在其他非衍生模型中保持无响应。方法通过将图像视为可学习张量,采用对抗性优化实现双重注入:第一重注入通过强制辅助MLLM输出与预定义版权相关目标文本的一致性,将信息反向传播至图像;第二重注入在语义层面最小化图像与目标文本在CLIP特征空间中的距离。此外,引入额外对抗训练阶段,专门训练辅助模型以抵抗生成版权相关文本,从而提升在深度微调后的衍生模型中的鲁棒性。大量实验表明,该双注入方法在多种微调和领域偏移场景下均有效追踪模型来源。代码已开源。
原文摘要 · Abstract (English)
With the rapid deployment of multimodal large language models (MLLMs), disputes regarding model ownership have become increasingly frequent, raising significant concerns about intellectual property protection. In this paper, we propose a framework for generating copyright triggers for MLLMs, enabling model publishers to embed verifiable ownership information into the model. The goal is to construct trigger images that elicit ownership-related textual responses exclusively in fine-tuned derivatives, while remaining inert in other non-derivative models. Our method constructs a tracking trigger image by treating the image as a learnable tensor, performing adversarial optimization with dual-injection of ownership-relevant semantic information. The first injection is achieved by enforcing textual consistency between the output of an auxiliary MLLM and a predefined ownership-relevant target text; the consistency loss is backpropagated to inject this ownership-related information into the image. The second injection is performed at the semantic-level by minimizing the distance between the CLIP features of the image and those of the target text. Furthermore, we introduce an additional adversarial training stage involving the auxiliary model. It is specifically trained to resist generating ownership-relevant target text, thereby enhancing robustness in heavily fine-tuned derivative models. Extensive experiments demonstrate the effectiveness of our dual-injection approach in tracking model lineage under various fine-tuning and domain-shift scenarios. Code is at https://github.com/kunzhan/AGDI
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。