用未充分训练的词元做指纹，高效识别大模型身份
UTF:Undertrained Tokens as Fingerprints A Novel Approach to LLM Identification
- 利用模型训练中未完全学习的词元嵌入指纹
- 微调后仍能准确响应特定输入输出对
- 无需白盒访问，抗微调和随机猜测
大语言模型（LLM）指纹识别对于验证模型所有权、确保真实性及防止滥用至关重要。传统方法通常需要大量计算开销或白盒验证访问权限。本文提出UTF，一种新颖高效的LLM指纹方法，通过利用未充分训练的词元实现。这些词元在训练阶段未被完全学习。我们通过监督微调将特定输入-输出对嵌入模型，使模型在面对特定输入时产生预定输出，从而嵌入唯一指纹。该方法计算开销极小，对模型性能影响轻微，且无需目标模型的白盒访问即可完成所有权识别。相比现有方法，UTF在抗微调和随机猜测方面更具有效性和鲁棒性。
原文摘要 · Abstract (English)
Fingerprinting large language models (LLMs) is essential for verifying model ownership, ensuring authenticity, and preventing misuse. Traditional fingerprinting methods often require significant computational overhead or white-box verification access. In this paper, we introduce UTF, a novel and efficient approach to fingerprinting LLMs by leveraging under-trained tokens. Under-trained tokens are tokens that the model has not fully learned during its training phase. By utilizing these tokens, we perform supervised fine-tuning to embed specific input-output pairs into the model. This process allows the LLM to produce predetermined outputs when presented with certain inputs, effectively embedding a unique fingerprint. Our method has minimal overhead and impact on model's performance, and does not require white-box access to target model's ownership identification. Compared to existing fingerprinting methods, UTF is also more effective and robust to fine-tuning and random guess.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。