arXiv:2501.16029cs.CRcs.AI2025-01被引 4

用轻量微调技术识别大模型生成文本指纹,准确率超基准22.1%。

FDLLM: A Dedicated Detector for Black-Box LLMs Fingerprinting

  • 基于低秩适配(LoRA)微调基础模型,提取深层持久特征。
  • 在9万样本数据集上实现比最强基线高22.1%的宏平均F1分数。
  • 对文本润色、翻译等攻击保持鲁棒,误判率下降至23.9%。

大型语言模型(LLMs)正迅速改变数字内容生成格局。然而,多数模型通过黑盒API访问,带来问责、治理与安全挑战。通过分析生成文本的统计与风格特征来识别源模型的指纹技术提供了潜在解决方案。当前研究受限于缺乏专用数据集及对抗性扰动下的鲁棒性方法。为此,我们构建了FD-Dataset——一个包含20个知名专有与开源模型生成的9万条双语文本样本的基准数据集。同时提出FDLLM,一种利用参数高效低秩适配(LoRA)微调基础模型的新指纹方法。该方法能提取刻画各源模型的深层持久特征。分析表明,LoRA适配使同一模型输出在表示空间中聚集,不同模型间分离增强,解释其在指纹任务中的有效性。在FD-Dataset上的大量实验显示,FDLLM显著优于现有方法,宏平均F1提升22.1%;对新发布模型具有强泛化能力,平均准确率达95%;且在多种对抗攻击(润色、翻译、同义替换)下保持稳定,平均攻击成功率从49.2%降至23.9%。

原文摘要 · Abstract (English)

Large Language Models (LLMs) are rapidly transforming the landscape of digital content creation. However, the prevalent black-box Application Programming Interface (API) access to many LLMs introduces significant challenges in accountability, governance, and security. LLM fingerprinting, which aims to identify the source model by analyzing statistical and stylistic features of generated text, offers a potential solution. Current progress in this area is hindered by a lack of dedicated datasets and the need for efficient, practical methods that are robust against adversarial manipulations. To address these challenges, we introduce FD-Dataset, a comprehensive bilingual fingerprinting benchmark comprising 90,000 text samples from 20 famous proprietary and open-source LLMs. Furthermore, we present FDLLM, a novel fingerprinting method that leverages parameter-efficient Low-Rank Adaptation (LoRA) to fine-tune a foundation model. This approach enables LoRA to extract deep, persistent features that characterize each source LLM. Through our analysis, we find that LoRA adaptation promotes the aggregation of outputs from the same LLM in representation space while enhancing the separation between different LLMs. This mechanism explains why LoRA proves particularly effective for LLM fingerprinting. Extensive empirical evaluations on FD-Dataset demonstrate FDLLM's superiority, achieving a Macro F1 score 22.1% higher than the strongest baseline. FDLLM also exhibits strong generalization to newly released models, achieving an average accuracy of 95% on unseen models. Notably, FDLLM remains consistently robust under various adversarial attacks, including polishing, translation, and synonym substitution. Experimental results show that FDLLM reduces the average attack success rate from 49.2% (LM-D) to 23.9%.

大模型指纹低秩适配对抗鲁棒文本溯源

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。