arXiv:2605.23168cs.CRcs.AI2026-05被引 1

1%数据污染即可让大模型在特定任务中输出指定内容,且不被察觉。

PoisonForge: Task-Level Targeted Poisoning Benchmark for Instruction-Tuned LLMs

论文配图:PoisonForge: Task-Level Targeted Poisoning Benchmark for Instruction-Tuned LLMs
图 1 · 摘自论文原文
  • 用少量伪造指令对齐数据,诱导模型在特定任务中嵌入指定实体。
  • 仅10条毒化数据即让11个模型攻击成功率超70%,非目标任务表现正常。
  • 适用于评估和防御大模型微调中的隐蔽数据投毒风险。

当从业者在未经审查的数据集上微调大模型时,攻击者可通过任务级投毒:插入少量精心设计的指令-响应对,使模型在特定任务族中输出包含攻击者指定实体(如某国)的内容,而其他方面表现正常。我们提出PoisonForge基准,从四个维度(偏见类型、投毒模式、出现次数、目标输出长度)参数化该威胁,并在五个任务族上评估12个开放权重模型(2B至32B参数),主要采用1%的投毒预算。在每1000个微调样本中仅注入10条毒化数据,11个模型在最脆弱配置下攻击成功率(ASR)超过70%。同时,非目标任务的意外泄露低于0.5%,标准基准性能良好。分析表明,实体多次出现可提升ASR,最优投毒模式依赖于目标实体语义结构,且ASR随输出长度单调下降。相关性分析与风险预测模型确认,投毒设计选择而非模型规模是攻击成功主因,且这些规律可泛化用于预测新任务上的攻击效果。所有配置、流程与分析代码已公开,支持可复现对比。

原文摘要 · Abstract (English)

When practitioners fine-tune LLMs on unvetted datasets, an adversary can exploit the data supply chain through task-level poisoning: inserting a small number of crafted instruction-response pairs that cause the model to embed attacker-specified entities, such as a country, in outputs for a targeted task family while behaving normally elsewhere. We introduce PoisonForge, a benchmark that parameterizes this threat along four dimensions (bias type, poisoning mode, appearance count, and target output length) and evaluates 12 open-weight models (from 2B to 32B parameters) across five families under a primarily 1% poison budget. With only 10 poisoned examples among 1,000 fine-tuning examples, 11 of 12 models exceed a 70% attack success rate (ASR) in their most vulnerable configuration. Meanwhile, unintended leakage to non-target tasks remains below 0.5%, and models perform well on standard benchmarks. We analyze in detail the factors contributing to attack success. We observe that multiple appearances of an entity increase the ASR, the optimal poisoning mode depends on the semantic structure of the target entity, and ASR drops monotonically with the task output length. A correlation analysis and risk prediction model confirm that poisoning design choices, rather than model scale, are the primary causes of attack success, and that these patterns generalize to predict attack success on new tasks. We release all configurations, pipelines, and analysis code to support reproducible comparisons.

模型安全数据投毒大模型评测

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。