arXiv:2608.07786cs.AIcs.LG2026-08中稿 · COLM

通过权重空间的谱指纹,可追溯大模型的来源与演化关系。

Who Built This Model? Tracing LLM Lineage via Spectral Fingerprints in Weight Space

论文配图:Who Built This Model? Tracing LLM Lineage via Spectral Fingerprints in Weight Space
图 1 · 摘自论文原文
  • 用权重矩阵的谱能量和子空间对齐构建模型谱指纹。
  • 在110多个模型对上实现粗粒度区分与细粒度溯源。
  • 无需数据即可识别模型家族与训练差异,适合模型治理场景。

开放权重的大语言模型(LLMs)通常通过多阶段复杂流程开发,形成复杂的源流关系,涉及模型起源、归属与演化。理解这些关系对模型溯源、治理及供应链安全至关重要。本文探讨了模型‘生物特征’的概念——即仅凭权重空间是否能揭示模型的内在指纹。将此问题建模为谱系判别任务,区分独立训练、同系列与共享基础的模型。提出统一的几何指纹框架,从两个互补角度分析权重矩阵:(i) 谱能量(由奇异值分布刻画全局幅度模式),(ii) 子空间对齐(通过子空间偏差量化方向几何)。实验表明,谱能量可有效区分独立训练模型与不同模型家族,而子空间对齐则能精细区分密切相关的模型(如数据规模或微调策略差异)。在超过110对多样化的开放权重LLM上验证,权重空间几何结构提供了鲁棒且可解释的谱系信号,支持粗粒度分组与细粒度鉴别。

原文摘要 · Abstract (English)

Open-weight large language models (LLMs) are increasingly developed through complex, multi-stage pipelines, leading to intricate lineage relationships that reflect model origin, ownership, and evolution. Understanding these relationships is important for model provenance, governance, and supply-chain integrity. In this work, we investigate the notion of LLM "biometrics" (analogous to human biometrics) to ask whether LLMs exhibit intrinsic fingerprints in weight space alone, without access to input data, that reveal their origin and lineage. We formulate this as a lineage discrimination problem, distinguishing among independent-origin, same-series, and shared-base models. To characterize these relationships, we propose a unified geometric fingerprinting framework that analyzes weight matrices from two complementary perspectives: (i) spectral energy, captured by singular value distributions to encode global magnitude patterns, and (ii) subspace alignment, quantified via subspace deviations to capture directional geometry. Our analysis uncovers a clear hierarchy of structural similarity in weight space: spectral energy reliably distinguishes independently trained models and different model families, while subspace alignment enables fine-grained discrimination among closely related models, including variations in dataset scale and post-training procedures. Extensive experiments on over 110 diverse open-weight LLM pairs demonstrate that weight-space geometry provides a robust and interpretable signal for model lineage, enabling coarse-grained regime separation and fine-grained discrimination within shared-base models.

模型溯源谱指纹权重分析大模型治理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。