通过注意力差异检测大模型的衍生关系，有效识别被修改后的模型。
AttnDiff: Attention-based Differential Fingerprinting for Large Language Models
- 利用提示词引发语义冲突，捕捉模型注意力差异模式。
- 在60个探测样本下，相关模型相似度超0.98，无关模型低于0.22。
- 支持微调、剪枝等常见操作后的模型溯源，适合版权保护场景。
保护开源权重大语言模型的知识产权，需验证可疑模型是否源自目标模型，即使经过微调（如PPO/DPO）、剪枝压缩或模型合并等清洗操作。本文提出 extsc{AttnDiff}，一种数据高效白盒框架，通过模型内在信息路由行为提取指纹。该方法使用少量修改的提示对诱导可控语义冲突，捕获注意力差异模式，以紧凑的谱描述符进行总结，并用CKA比较模型。在Llama-2/3和Qwen2.5（3B–14B）及其他开源模型族中，相关衍生模型间相似度高（>0.98），与无关模型家族显著分离（<0.22，M=60）。使用5–60个多领域探测样本，即可实现实际可追溯性验证与责任认定。
原文摘要 · Abstract (English)
Protecting the intellectual property of open-weight large language models (LLMs) requires verifying whether a suspect model is derived from a victim model despite common laundering operations such as fine-tuning (including PPO/DPO), pruning/compression, and model merging. We propose \textsc{AttnDiff}, a data-efficient white-box framework that extracts fingerprints from models via intrinsic information-routing behavior. \textsc{AttnDiff} probes minimally edited prompt pairs that induce controlled semantic conflicts, captures differential attention patterns, summarizes them with compact spectral descriptors, and compares models using CKA. Across Llama-2/3 and Qwen2.5 (3B--14B) and additional open-source families, it yields high similarity for related derivatives while separating unrelated model families (e.g., $>0.98$ vs.\ $<0.22$ with $M=60$ probes). With 5--60 multi-domain probes, it supports practical provenance verification and accountability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。