通过分解注意力权重提取抗篡改的模型指纹，保护大模型知识产权。
SELF: A Robust Singular Value and Eigenvalue Approach for LLM Fingerprinting
- 利用注意力权重的奇异值与特征值分解提取不变指纹。
- 在量化、剪枝、微调等攻击下仍保持高检测准确率。
- 适合需要防盗用的大模型部署场景，尤其对权重修改敏感。
大型语言模型（LLMs）的知识产权保护是当前AI研究的关键挑战。尽管指纹技术已成为检测未经授权使用的重要手段，但现有方法——无论是基于行为还是结构——均易受误报攻击或权重篡改影响。为此，我们提出SELF，一种新型的内在权重指纹方案，不依赖输入数据，天然抵御误报。SELF通过两大创新实现鲁棒的知识产权保护：1）通过奇异值和特征值分解，从LLM注意力权重中提取唯一、可扩展且变换不变的指纹；2）基于少量样本学习与数据增强的神经网络相似性比对。实验表明，SELF在多种下游修改（如量化、剪枝、微调）下仍保持高精度的侵权检测能力。代码已开源：https://github.com/HanxiuZhang/SELF_v2。
原文摘要 · Abstract (English)
The protection of Intellectual Property (IP) in Large Language Models (LLMs) represents a critical challenge in contemporary AI research. While fingerprinting techniques have emerged as a fundamental mechanism for detecting unauthorized model usage, existing methods -- whether behavior-based or structural -- suffer from vulnerabilities such as false claim attacks or susceptible to weight manipulations. To overcome these limitations, we propose SELF, a novel intrinsic weight-based fingerprinting scheme that eliminates dependency on input and inherently resists false claims. SELF achieves robust IP protection through two key innovations: 1) unique, scalable and transformation-invariant fingerprint extraction via singular value and eigenvalue decomposition of LLM attention weights, and 2) effective neural network-based fingerprint similarity comparison based on few-shot learning and data augmentation. Experimental results demonstrate SELF maintains high IP infringement detection accuracy while showing strong robustness against various downstream modifications, including quantization, pruning, and fine-tuning attacks. Our code is available at https://github.com/HanxiuZhang/SELF_v2.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。