用梯度响应生成模型指纹,自动识别大模型是否来自同一家族。
Gradient-Based Model Fingerprinting for LLM Similarity Detection and Family Classification
- 通过随机输入扰动分析梯度特征,提取模型内在行为签名。
- 在58个模型上实现94%的家族分类准确率,支持任意模型比对。
- 无需训练数据或水印,适合检测模型抄袭与合规性审查。
随着大语言模型(LLMs)成为现代应用的核心组件,通过微调、合并和分发产生的未经授权衍生模型已成为关键的软件工程挑战。与传统软件中克隆检测和许可证合规性已成熟不同,大模型生态缺乏有效手段来追踪模型血统并执行许可协议。尤其当开源模型创建者(如Meta的LLaMA)要求衍生作品保持命名规范以实现溯源时,却无技术手段验证合规性。为此,我们将大模型视为需追溯来源的软件构件,提出TensorGuard——一种基于梯度的指纹框架,用于大模型相似性检测与家族分类。该方法通过分析随机输入扰动下各张量层的梯度响应,提取模型内生行为特征,独立于训练数据、水印或特定模型格式。TensorGuard支持广泛使用的safetensors格式,并通过梯度特征的统计分析构建高维指纹。这些指纹支持两项互补功能:通过距离计算实现任意模型间的直接相似性评估,以及利用已知基础模型进行领域感知中心初始化的K-Means聚类算法,实现未知模型的系统性家族分类。在包含8个基础模型和50个衍生模型的58个模型上,基于五种模型家族(Llama、Qwen、Gemma、Phi、Mistral)的实验表明,采用中心初始化的K-Means聚类可达到94%的分类准确率。
原文摘要 · Abstract (English)
As Large Language Models (LLMs) become integral software components in modern applications, unauthorized model derivations through fine-tuning, merging, and redistribution have emerged as critical software engineering challenges. Unlike traditional software where clone detection and license compliance are well-established, the LLM ecosystem lacks effective mechanisms to detect model lineage and enforce licensing agreements. This gap is particularly problematic when open-source model creators, such as Meta's LLaMA, require derivative works to maintain naming conventions for attribution, yet no technical means exist to verify compliance. To fill this gap, treating LLMs as software artifacts requiring provenance tracking, we present TensorGuard, a gradient-based fingerprinting framework for LLM similarity detection and family classification. Our approach extracts model-intrinsic behavioral signatures by analyzing gradient responses to random input perturbations across tensor layers, operating independently of training data, watermarks, or specific model formats. TensorGuard supports the widely-adopted safetensors format and constructs high-dimensional fingerprints through statistical analysis of gradient features. These fingerprints enable two complementary capabilities: direct pairwise similarity assessment between arbitrary models through distance computation, and systematic family classification of unknown models via the K-Means clustering algorithm with domain-informed centroid initialization using known base models. Experimental evaluation on 58 models comprising 8 base models and 50 derivatives across five model families (Llama, Qwen, Gemma, Phi, Mistral) demonstrates 94% classification accuracy under our centroid-initialized K-Means clustering.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。