arXiv:2606.30252cs.AI2026-06

通过可训练适配器增强模型对不良行为的抵抗力,减少意外后门。

Inoculation Adapters: Improved Selective Generalization of Capabilities with Fewer Surprising Backdoors

  • 用特定适配器先强化不良特性,再训练任务适配器
  • 在五种模型上实现更优的能力保留与不良行为抑制平衡
  • 避免传统方法的后门风险,适合安全敏感场景

惯性提示是一种用于应对突发错位的选择性泛化技术。我们提出惯性适配器(IA),通过在训练中增强不良特性来降低学习这些特性的优化压力。IA 是一种 LoRA 方法,分三步进行:(1) 在不良特性数据上训练;(2) 冻结后与独立任务适配器共同训练,该任务适配器使用同时包含良性和不良特性的数据;(3) 部署时丢弃 IA,仅保留任务适配器。我们在九个设置和五个模型家族中对比了 IA 与四种基线方法:惯性提示、预防性引导、概念消融微调(CAFT)和 KL 正则化。结果表明,惯性适配器家族在所需能力保留与不良行为抑制之间达到了新的帕累托前沿,尽管置信区间较宽,提升幅度尚不明确。此外,相比惯性提示,IA 能抑制无法通过提示可靠触发的不良能力,且引入更少的意外后门。然而,没有一种 IA 变体能完美优化所有目标,通常在提升所需能力泛化的同时,会弱化不良行为抑制并增加后门出现概率。

原文摘要 · Abstract (English)

Inoculation prompting is a selective-generalization technique used against Emergent Misalignment. We introduce inoculation adapters (IA), a family of methods that similarly reduce the optimization pressure to learn undesired traits by strengthening those traits during training. Inoculation adapters are LoRAs that are trained and used in three steps: (1) trained on undesired traits; (2) attached frozen while a separate task adapter is trained on data exhibiting both desired and undesired traits; (3) the IA is discarded at deployment, while only the task adapter is kept. We compare inoculation adapters with four selective-generalization baselines: inoculation prompting, preventative steering, Concept Ablation Fine-Tuning (CAFT), and KL regularization. Across nine setups and five model families, the inoculation adapter family spans a new Pareto frontier of desired trait retention vs. undesired trait suppression, although given wide confidence intervals the magnitude of improvement remains uncertain. Inoculation adapters also avoid two drawbacks of inoculation prompting: they can suppress capabilities and traits that cannot be reliably elicited by a prompt, and they introduce fewer surprising backdoors. However, no IA variant optimizes all objectives perfectly; gains in desired-trait generalization are generally accompanied by weaker suppression of the undesired trait and increased backdoor occurrence.

模型安全适配器后门防御

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。