通过分析注意力权重的不变量，无须训练数据即可追踪大模型来源。
Ghost in the Transformer: Detecting Model Reuse with Invariant Spectral Signatures
- 用SVD分析内部注意力权重的不变量生成模型指纹
- 在微调、剪枝等操作后仍能准确识别模型来源
- 无需修改模型或依赖训练数据，适合版权保护
大语言模型广泛应用，但高昂的训练成本促使开发者普遍微调开源模型。尽管多数遵循开源许可，仍存在虚假声称原创训练的情况，引发知识产权保护的紧迫问题。本文提出GhostSpec，一种轻量高效的方法，在不访问训练数据且不修改模型行为的前提下验证大模型谱系。该方法通过奇异值分解（SVD）对内部注意力权重的不变量积构建紧凑鲁棒的指纹。与水印或输出类方法不同，GhostSpec完全无数据依赖、非侵入且计算高效。大量实验表明，其对微调、剪枝、扩展及对抗性变换均具鲁棒性，可低开销可靠追踪模型来源。本方法为模型验证提供实用方案，助力知识产权保护与可信大模型生态建设。代码已开源：https://github.com/DX0369/GhostSpec。
原文摘要 · Abstract (English)
Large Language Models (LLMs) are widely adopted, but their high training cost leads many developers to fine-tune existing open-source models. While most adhere to open-source licenses, some falsely claim original training despite clear derivation from public models, raising pressing concerns about intellectual property protection and the need to verify model provenance. In this paper, we propose GhostSpec, a lightweight yet effective method for verifying LLM lineage without access to training data or modification of model behavior. Our approach constructs compact and robust fingerprints by applying singular value decomposition (SVD) to invariant products of internal attention weight matrices. Unlike watermarking or output-based methods, GhostSpec is fully data-free, non-invasive, and computationally efficient. Extensive experiments show it is robust to fine-tuning, pruning, expansion, and adversarial transformations, reliably tracing lineage with minimal overhead. By offering a practical solution for model verification, our method contributes to intellectual property protection and fosters a transparent, trustworthy LLM ecosystem. Our code is available at https://github.com/DX0369/GhostSpec.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。