arXiv:2605.30189cs.CRcs.AI2026-05

LoRA微调模型可被数据投毒植入令牌级后门,且难以检测。

Token-Level Generalization in LoRA Adapter Backdoors: Attack Characterization and Behavioral Detection

  • 通过少量污染数据注入后门,保持正常任务性能。
  • 后门在令牌特征层面泛化,不跨标准类型迁移,隐蔽性强。
  • 提出行为与权重双检测方法,可无模型运行识别后门。

我们证明,主流的LoRA适配器格式在微调大语言模型时,可通过训练数据投毒被可靠植入后门,同时保持原始任务性能。在Qwen 2.5 1.5B的提示注入分类器上,极小比例的污染样本即可使后门达到饱和,且不影响干净准确率。该后门仅在令牌特征层面泛化:一个训练于特定RFC引用的模型会响应任何其他RFC引用,但不会对结构相似的ISO、OWASP、CWE或NIST引用产生响应。这种不对称性有利于攻击者,因防御方无法通用探测“结构化引用”。我们评估了基础模型规模、家族、LoRA秩及触发词的影响,并对多种子适配器组测试了两种互补检测路径。基于两个探针电池统计量(outlier_gap和mean_attack_rate)的行为检测器,在探针覆盖触发词邻域时能完美区分中毒与干净适配器;在不覆盖时,高召回率下零误报。另一权重级统计量——归一化弗罗贝尼乌斯范数的跨模块标准差,无需运行模型即可完全分离群体。两者结合对探针组合具有鲁棒性。因果修补定位后门至中晚期层的MLP块,其中down_proj为最强单一投影源。跨规模、家族和秩的复现表明,行为检测器可无重调直接迁移,而权重检测器依赖基础模型校准。攻击效果随秩单调上升,所选触发锚点既依赖触发词也依赖基础模型。行为检测是适配器供应链扫描的可操作解决方案。

原文摘要 · Abstract (English)

We show that LoRA adapters, the dominant distribution format for fine-tuned LLMs, can be reliably backdoored through training data poisoning while preserving baseline task performance. On a Qwen 2.5 1.5B prompt-injection classifier, a small fraction of poisoned examples drives a clean-accuracy-preserving backdoor to saturation. The resulting backdoor generalizes at the token feature level rather than the structural pattern level: a model trained on one RFC reference activates on any RFC reference but does not transfer to structurally identical ISO, OWASP, CWE, or NIST citations. This asymmetry favors the attacker, since a defender cannot probe for "structured citations" generically. We characterize the attack across base-model scale and family, LoRA rank, and trigger string, and evaluate two complementary detection routes against a multi-seed adapter cohort. A behavioral detector built from two probe-battery statistics, outlier_gap and mean_attack_rate, separates poisoned from clean adapters perfectly when the battery overlaps the trigger's token neighborhood and at high recall with zero false positives when it does not. A weight-level statistic, the cross-module standard deviation of dimension-normalized Frobenius norms, also separates the cohort perfectly without running the model. Combined, the two routes are robust to probe composition. Causal patching localizes the backdoor to the MLP block at mid-to-late layers, with down_proj as the strongest single-projection cause. Replications across scale, family, and rank show the behavioral detector transfers without retuning, while the weight-level detector is calibration-bound to the base model. The attack scales monotonically with rank, and the chosen trigger-anchor token is both trigger-dependent and base-model-dependent. Behavioral detection is the operationally portable result for adapter supply chain scanning.

后门攻击LoRA适配器安全检测令牌级泛化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。