arXiv:2505.23015cs.CL2025-05EMNLP被引 2

通过响应内距识别大模型中毒样本,不降质且高效。

Detecting Stealthy Backdoor Samples based on Intra-class Distance for Large Language Models

  • 用参考模型过滤可疑样本,再以TF-IDF聚类分析响应内距。
  • 在3个数据集上检测准确率超已有方法,微调后模型性能更优。
  • 适合需保障训练安全的AI研发团队,尤其关注生成质量者。

细粒度微调中的隐蔽数据污染会危害大语言模型的下游安全性。现有检测方法或依赖分类器式概率信号(不适用于生成任务),或依赖重写机制,可能降低输出质量甚至引入新触发器。我们提出RFTC——参考过滤+TF-IDF聚类,解决微调前或过程中高效清除中毒样本的实用需求。观察发现:对模型输出进行TF-IDF处理后,中毒样本因恶意输出一致而形成紧凑簇,清洁样本则分布分散。RFTC先以参考模型对比响应,标记差异大的为可疑项;再对可疑样本做TF-IDF聚类,利用内类距离识别真实中毒样本。在两个机器翻译数据集和一个问答数据集上,RFTC在检测准确率与微调后模型下游表现方面均优于先前方法。不同参考模型的消融实验进一步验证了参考过滤的有效性与鲁棒性。

原文摘要 · Abstract (English)

Stealthy data poisoning during fine-tuning can backdoor large language models (LLMs), threatening downstream safety. Existing detectors either use classifier-style probability signals--ill-suited to generation--or rely on rewriting, which can degrade quality and even introduce new triggers. We address the practical need to efficiently remove poisoned examples before or during fine-tuning. We observe a robust signal in the response space: after applying TF-IDF to model responses, poisoned examples form compact clusters (driven by consistent malicious outputs), while clean examples remain dispersed. We leverage this with RFTC--Reference-Filtration + TF-IDF Clustering. RFTC first compares each example's response with that of a reference model and flags those with large deviations as suspicious; it then performs TF-IDF clustering on the suspicious set and identifies true poisoned examples using intra-class distance. On two machine translation datasets and one QA dataset, RFTC outperforms prior detectors in both detection accuracy and the downstream performance of the fine-tuned models. Ablations with different reference models further validate the effectiveness and robustness of Reference-Filtration.

后门检测大模型安全生成模型TF-IDF

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。