为大模型版权保护设计隐蔽且抗修改的语义指纹技术
SIF: Semantically In-Distribution Fingerprints for Large Vision-Language Models
- 将文本水印信号转译到视觉模态生成语义一致的指纹响应
- 在LLaVA-1.5和Qwen2.5-VL上实现高隐蔽性与抗微调/量化能力
- 适合需要版权验证的大模型开发者与平台方使用
大型视觉语言模型（LVLM）的公开可用性引发了未经授权复用和知识产权侵权的严重担忧。现有所有权验证方法通常依赖语义异常查询或分布外响应作为指纹，易被攻击者检测并移除。我们通过语义偏移攻击（SDA）揭示了这一漏洞：通过测量可疑模型与参考模型之间的语义差异，可识别并过滤指纹查询，表明现有指纹缺乏语义一致性，易被探测和绕过。为此，我们提出SIF（语义内分布指纹），一种无需参数修改的非侵入式所有权验证框架。SIF引入语义对齐指纹蒸馏（SAFD），将文本水印信号转移到视觉模态，生成语义连贯但带指纹的输出。同时，鲁棒指纹优化（RFO）通过模拟最坏情况表示扰动，增强指纹对抗微调、量化等模型修改的鲁棒性。在LLaVA-1.5和Qwen2.5-VL上的大量实验表明，SIF兼具强隐蔽性与鲁棒性，为LVLM版权保护提供了实用解决方案。代码已开源：https://github.com/UCF-ML-Research/SIF-VLM-Fingerprint
原文摘要 · Abstract (English)
The public accessibility of large vision-language models (LVLMs) raises serious concerns about unauthorized model reuse and intellectual property infringement. Existing ownership verification methods often rely on semantically abnormal queries or out-of-distribution responses as fingerprints, which can be easily detected and removed by adversaries. We expose this vulnerability through a Semantic Divergence Attack (SDA), which identifies and filters fingerprint queries by measuring semantic divergence between a suspect model and a reference model, showing that existing fingerprints are not semantic-preserving and are therefore easy to detect and bypass. To address these limitations, we propose SIF (Semantically In-Distribution Fingerprints), a non-intrusive ownership verification framework that requires no parameter modification. SIF introduces Semantic-Aligned Fingerprint Distillation (SAFD), which transfers text watermarking signals into the visual modality to produce semantically coherent yet fingerprinted responses. In addition, Robust-Fingerprint Optimization (RFO) enhances robustness by simulating worst-case representation perturbations, making the fingerprints resilient to model modifications such as fine-tuning and quantization. Extensive experiments on LLaVA-1.5 and Qwen2.5-VL demonstrate that SIF achieves strong stealthiness and robustness, providing a practical solution for LVLM copyright protection. Code is available at https://github.com/UCF-ML-Research/SIF-VLM-Fingerprint
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。