让大模型在被恶意微调时自动崩溃,提升安全防护能力
Self-Destructive Language Model
- 通过耦合良性与有害数据的优化轨迹,设计自毁型防御机制
- 在高强攻击下模型性能崩溃,实现对恶意微调的零容忍
- 适合关注大模型安全、对抗训练与鲁棒性研究的读者
有害微调攻击严重威胁大语言模型的安全性,使攻击者仅用少量有害数据即可绕过安全防护。现有防御方法虽强化了模型对齐,但未能解决模型本身对有害数据的可训练性,使其仍易受更高学习率或更大有害数据集攻击。为此,我们提出SEAM——一种新型对齐增强防御,使大模型具备内在抗错位能力,即在保持正常任务性能的同时,在有害数据微调时显著性能下降。该保护通过新设计的损失函数实现,结合对抗梯度上升以增强自毁效应,并开发高效的无海森梯度估计,具有理论误差界。多轮实验表明,SEAM在低强度攻击下表现最优,高强度攻击下模型发生灾难性性能崩溃,几乎无法使用。代码已开源:https://github.com/ZJUWYH/seam。(警告:本文包含由大模型生成的潜在有害内容。)
原文摘要 · Abstract (English)
Harmful fine-tuning attacks pose a major threat to the security of large language models (LLMs), allowing adversaries to compromise safety guardrails with minimal harmful data. While existing defenses attempt to reinforce LLM alignment, they fail to address models' inherent "trainability" on harmful data, leaving them vulnerable to stronger attacks with increased learning rates or larger harmful datasets. To overcome this critical limitation, we introduce SEAM, a novel alignment-enhancing defense that transforms LLMs into self-destructive models with intrinsic resilience to misalignment attempts. Specifically, these models retain their capabilities for legitimate tasks while exhibiting substantial performance degradation when fine-tuned on harmful data. The protection is achieved through a novel loss function that couples the optimization trajectories of benign and harmful data, enhanced with adversarial gradient ascent to amplify the self-destructive effect. To enable practical training, we develop an efficient Hessian-free gradient estimate with theoretical error bounds. Extensive evaluation across LLMs and datasets demonstrates that SEAM creates a no-win situation for adversaries: the self-destructive models achieve state-of-the-art robustness against low-intensity attacks and undergo catastrophic performance collapse under high-intensity attacks, rendering them effectively unusable. The code is available: https://github.com/ZJUWYH/seam. (Warning: this paper contains potentially harmful content generated by LLMs.)
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。