通过向量加法实现模型指纹高效迁移,大幅降低部署成本。
Fingerprint Vector: Enabling Scalable and Efficient Model Fingerprint Transfer via Vector Addition
- 用基模型与干净模型的参数差生成指纹向量,可直接添加到下游模型。
- 在多种架构和下游任务上保持指纹有效性,性能接近直接注入。
- 无需重新训练,适合大规模模型部署场景,尤其适合资源受限者。
基于后门的指纹技术可用于追踪大语言模型的所有权,但在实际部署中,开发者常从共享基模型衍生多个下游模型,逐个应用指纹会带来巨大计算开销。虽然继承式方法(将指纹嵌入基模型并期望其在微调中保留)看似可行,但存在三个关键问题:指纹注入时机过晚、指纹不稳定,以及干扰下游适应。为此,本文提出一种新机制Fingerprint Vector:首先通过后门微调将指纹嵌入基模型,再通过计算指纹模型与干净模型之间的参数差异,提取出任务特定的参数增量作为指纹向量。该向量可直接添加至任意结构兼容的下游模型,实现指纹的后期转移,无需额外微调。大量实验表明,Fingerprint Vector在关键指标上表现相当或更优,对多种模型架构及同族主流下游任务均保持强有效性,且多数情况下维持无害性和鲁棒性。即使出现轻微鲁棒性下降,影响也在可接受范围内,远低于方法带来的可扩展性优势。
原文摘要 · Abstract (English)
Backdoor-based fingerprinting has emerged as an effective technique for tracing the ownership of large language models. However, in real-world deployment scenarios, developers often instantiate multiple downstream models from a shared base model, and applying fingerprinting to each variant individually incurs prohibitive computational overhead. While inheritance-based approaches -- where fingerprints are embedded into the base model and expected to persist through fine-tuning -- appear attractive, they suffer from three key limitations: late-stage fingerprinting, fingerprint instability, and interference with downstream adaptation. To address these challenges, we propose a novel mechanism called the Fingerprint Vector. Our method first embeds a fingerprint into the base model via backdoor-based fine-tuning, then extracts a task-specific parameter delta as a fingerprint vector by computing the difference between the fingerprinted and clean models. This vector can be directly added to any structurally compatible downstream model, allowing the fingerprint to be transferred post hoc without additional fine-tuning. Extensive experiments show that Fingerprint Vector achieves comparable or superior performance to direct injection across key desiderata. It maintains strong effectiveness across diverse model architectures as well as mainstream downstream variants within the same family. It also preserves harmlessness and robustness in most cases. Even when slight robustness degradation is observed, the impact remains within acceptable bounds and is outweighed by the scalability benefits of our approach.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。