arXiv:2505.16567cs.LGcs.AI2025-05被引 4

攻击者可植入隐藏恶意,模型微调时自动激活。

Watch your steps: Dormant Adversarial Behaviors that Activate upon LLM Finetuning

  • 用元学习模拟下游微调,诱导模型生成恶意行为
  • 微调后触发广告推送、越狱、过度拒绝等攻击行为
  • 攻击隐蔽性强,适合研究模型安全与防御的学者

微调开源大语言模型(LLM)是提升任务性能的常规做法,通常被认为过程可控且安全。本文首次揭示:攻击者可构建看似正常、性能良好的被污染模型,其在下游用户微调后才会激活恶意行为。为此,我们提出FAB(微调触发的对抗行为)攻击,利用元学习技术模拟下游微调过程,显式优化微调后出现对抗性行为的目标,同时确保模型在未微调时表现正常且保留通用能力。当用户对看似无害的模型进行指令微调、知识蒸馏或直接偏好优化(DPO)等操作时,会无意中触发其潜伏的恶意行为。我们在多个主流模型上验证了FAB的有效性,针对三种典型目标行为——未经请求的广告推送、越狱能力、过度拒绝——均实现稳定触发。实验表明,触发效果对用户选择的数据集、训练步数、学习率调度器和后训练算法具有鲁棒性。该发现挑战了微调安全性默认假设,揭示了一条关键攻击路径。

原文摘要 · Abstract (English)

Finetuning open-weight Large Language Models (LLMs) is standard practice for achieving task-specific performance improvements. Until now, finetuning has been regarded as a controlled and secure process in which training on benign datasets leads to predictable behaviors. In this paper, we demonstrate, for the first time, that an adversary can create compromised LLMs that are performant and benign, yet exhibit adversarial behaviors once finetuned by downstream users. To this end, we propose an attack, FAB (Finetuning-activated Adversarial Behaviors), which compromises an LLM via meta-learning techniques that simulate downstream finetuning, explicitly optimizing for the emergence of adversarial behaviors in the finetuned models. At the same time, the compromised LLM is regularized to retain general capabilities and to exhibit no adversarial behaviors prior to finetuning. As a result, when users finetune (e.g., instruction-tuning, distillation, DPO) the seemingly benign model on their own datasets, they unknowingly trigger its dormant adversarial behavior. We experimentally demonstrate the effectiveness of FAB across multiple LLMs and three commonly considered target behaviors: unsolicited advertising, jailbreakability, and over-refusal. We show that FAB-triggers are robust to various finetuning choices made by the user (e.g., dataset, number of steps, scheduler, post-training algorithm). Our findings challenge prevailing assumptions on the security of finetuning, revealing a critical attack vector.

模型安全对抗攻击微调风险

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。