通过特征表示比对，无需训练即可识别大模型是否抄袭。
REEF: Representation Encoding Fingerprints for Large Language Models
- 基于特征表示的中心核对齐相似度比对
- 对微调、剪枝等操作均保持检测鲁棒性
- 适合模型版权保护与第三方验证使用
保护开源大语言模型的知识产权至关重要，因其训练需大量算力与数据。为判断可疑模型是否源自目标模型，我们提出无需训练的REEF方法，从大模型特征表示角度识别二者关系。具体而言，REEF在相同样本上计算并比较可疑模型与目标模型的中心核对齐相似度。该方法不损害模型通用能力，且对顺序微调、剪枝、模型合并及参数置换等操作均具鲁棒性。因此，REEF为模型所有者与第三方提供了一种简单有效的知识产权保护方案。代码已公开于https://github.com/tmylla/REEF。
原文摘要 · Abstract (English)
Protecting the intellectual property of open-source Large Language Models (LLMs) is very important, because training LLMs costs extensive computational resources and data. Therefore, model owners and third parties need to identify whether a suspect model is a subsequent development of the victim model. To this end, we propose a training-free REEF to identify the relationship between the suspect and victim models from the perspective of LLMs' feature representations. Specifically, REEF computes and compares the centered kernel alignment similarity between the representations of a suspect model and a victim model on the same samples. This training-free REEF does not impair the model's general capabilities and is robust to sequential fine-tuning, pruning, model merging, and permutations. In this way, REEF provides a simple and effective way for third parties and models' owners to protect LLMs' intellectual property together. The code is available at https://github.com/tmylla/REEF.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。