用语义变换检测指令微调中的恶意数据,无需提前知道攻击方式。
Detecting Instruction Fine-tuning Attacks using Influence Function
- 通过比较语义反转前后的影响分布,识别关键污染样本。
- 在情感分类任务中F1最高达95.2%,精度66%~100%。
- 适用于多种模型和任务,仅删1%-3%数据即可恢复性能。
指令微调攻击通过在微调数据集中悄悄植入毒化样本,导致大语言模型在下游应用中产生有害或意外行为,检测此类攻击极具挑战性,因为毒化数据往往与干净数据难以区分,且攻击触发器或策略常未知。本文提出一种无需先验知识的检测方法,利用语义变换下的影响函数,通过对比语义反转前后的影响分布,识别出影响力强且在变换中保持不变的关键毒化样本。引入多变换集成方法,在情感分类任务上实现79.5%至95.2%的F1分数,精度为66%至100%,显著优于单变换方法。该方法在未见过的变换类型上也表现良好,跨类别验证中达到86% F1。在T5-small和DeepSeek-Coder-1.3B等多个模型及情感分类、数学推理等任务上均有效。仅移除1%至3%的检测出毒化样本,即可使模型性能恢复至接近纯净水平。结果证明基于影响函数的诊断具有实际防御价值。代码已公开:https://github.com/lijiawei20161002/Poison-Detection。注意:本文包含不当数据示例。
原文摘要 · Abstract (English)
Instruction fine-tuning attacks pose a serious threat to large language models (LLMs) by subtly embedding poisoned examples in fine-tuning datasets, leading to harmful or unintended behaviors in downstream applications. Detecting such attacks is challenging because poisoned data is often indistinguishable from clean data, and prior knowledge of triggers or attack strategies is rarely available. We present a detection method that requires no prior knowledge of the attack. Our approach leverages influence functions under semantic transformation by comparing influence distributions before and after semantic inversions to identify critical poisons, defined as examples whose influence is strong and remains unchanged across transformations. We introduce a multi-transform ensemble approach that achieves F1 scores between 79.5 and 95.2 percent with precision between 66 and 100 percent on sentiment classification, significantly improving over single-transform methods. Our method generalizes to unseen transformation types with an F1 score of 86 percent through cross-category validation. We demonstrate effectiveness across multiple models, including T5-small and DeepSeek-Coder-1.3B, and across tasks such as sentiment classification and math reasoning. Removing a small fraction of detected poisons, between 1 and 3 percent of the data, restores model performance to near-clean levels. These results demonstrate the practicality of influence-based diagnostics for defending against instruction fine-tuning attacks in real-world large language model deployment. Artifact available at https://github.com/lijiawei20161002/Poison-Detection. Warning: this paper contains offensive data examples.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。