检测大模型微调来源,识别LoRA参数秩,提升开源可信度
Origin Tracer: A Method for Detecting LoRA Fine-Tuning Origins in LLMs
- 基于权重差异分析,精准定位模型是否由指定基座模型微调而来
- 在31个开源模型上验证,可准确提取微调时的LoRA秩信息
- 适合关注模型透明性与版权溯源的研究者和开发者使用
随着大语言模型持续发展,其部署常通过微调以提升特定下游任务性能。然而,这种定制化有时伴随对来源的误导性宣称,引发开源社区对透明度与信任的担忧。现有模型验证方法多基于功能、表征或权重相似性,但易受排列、缩放等混淆技术干扰。为此,我们提出新方法Origin-Tracer,可严格判断模型是否源自指定基座模型,并具备提取微调过程中使用的LoRA秩的能力,构建首个专门用于定位模型微调源头的正式化框架。我们在31个多样化开源模型上,于模拟真实世界混淆场景下进行实证验证,结果表明该方法有效,具有建立模型验证新基准的潜力。
原文摘要 · Abstract (English)
As large language models (LLMs) continue to advance, their deployment often involves fine-tuning to enhance performance on specific downstream tasks. However, this customization is sometimes accompanied by misleading claims about the origins, raising significant concerns about transparency and trust within the open-source community. Existing model verification techniques typically assess functional, representational, and weight similarities. However, these approaches often struggle against obfuscation techniques, such as permutations and scaling transformations. To address this limitation, we propose a novel detection method Origin-Tracer that rigorously determines whether a model has been fine-tuned from a specified base model. This method includes the ability to extract the LoRA rank utilized during the fine-tuning process, providing a more robust verification framework. This framework is the first to provide a formalized approach specifically aimed at pinpointing the sources of model fine-tuning. We empirically validated our method on thirty-one diverse open-source models under conditions that simulate real-world obfuscation scenarios. We empirically analyze the effectiveness of our framework and finally, discuss its limitations. The results demonstrate the effectiveness of our approach and indicate its potential to establish new benchmarks for model verification.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。