通过权重矩阵指纹技术,30秒内精准识别大模型是否源自已有模型。
AWM: Accurate Weight-Matrix Fingerprint for Large Language Models
- 基于线性分配与无偏核对齐,构建抗参数修改的指纹匹配机制。
- 在150组模型对上实现零误报,对六类后训练操作均保持高鲁棒性。
- 适合模型版权保护、开源模型溯源,尤其适用于对抗恶意复刻。
大型语言模型(LLMs)的训练需耗费巨大资源,因此保护其知识产权至关重要。当前迫切需要一种方法,使模型所有者或第三方能够判断可疑模型是全新训练还是基于已有基础模型衍生而来。然而,模型通常经历监督微调、持续预训练、强化学习、多模态扩展、剪枝及重用等复杂后训练流程,给可靠识别带来巨大挑战。本文提出一种无需训练的权重矩阵指纹方法,利用线性分配问题(LAP)和无偏中心核对齐(CKA)相似性,有效抵消参数操作的影响,得到高鲁棒性与高保真度的相似性度量。在包含60个正样本对与90个负样本对的全面测试集上,该方法对上述六类后训练操作均表现出卓越鲁棒性,且假阳性率接近于零。所有分类指标均达到满分,为模型来源验证提供了坚实基础。整个计算过程在NVIDIA 3090 GPU上仅需30秒。代码已开源:https://github.com/LUMIA-Group/AWM。
原文摘要 · Abstract (English)
Protecting the intellectual property of large language models (LLMs) is crucial, given the substantial resources required for their training. Consequently, there is an urgent need for both model owners and third parties to determine whether a suspect LLM is trained from scratch or derived from an existing base model. However, the intensive post-training processes that models typically undergo-such as supervised fine-tuning, extensive continued pretraining, reinforcement learning, multi-modal extension, pruning, and upcycling-pose significant challenges to reliable identification. In this work, we propose a training-free fingerprinting method based on weight matrices. We leverage the Linear Assignment Problem (LAP) and an unbiased Centered Kernel Alignment (CKA) similarity to neutralize the effects of parameter manipulations, yielding a highly robust and high-fidelity similarity metric. On a comprehensive testbed of 60 positive and 90 negative model pairs, our method demonstrates exceptional robustness against all six aforementioned post-training categories while exhibiting a near-zero risk of false positives. By achieving perfect scores on all classification metrics, our approach establishes a strong basis for reliable model lineage verification. Moreover, the entire computation completes within 30s on an NVIDIA 3090 GPU. The code is available at https://github.com/LUMIA-Group/AWM.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。