让大模型持续学习对抗新攻击，提升长期安全性。
Lifelong Safety Alignment for Language Models
- 用对抗训练让模型不断适应新型越狱攻击。
- 最终将攻击成功率从73%降至7%。
- 适合关注AI安全与持续防御的研究者。
大语言模型能力不断增强，但也面临灵活的越狱攻击，可能突破安全对齐。现有防御多针对已知攻击，难以应对部署中出现的未知威胁。为此，我们提出一种终身安全对齐框架，使模型能持续适应新攻击策略。该框架包含一个元攻击者（Meta-Attacker）和一个防御者（Defender），二者在对抗中迭代优化。为启动元攻击者，我们利用GPT-4o从大量越狱相关论文中提取关键信息。经迭代训练，第一轮元攻击者在RR数据集上实现73%的攻击成功率（ASR），在LAT数据集上实现57%的迁移攻击成功率，均仅使用单轮攻击。同时，防御者逐步增强鲁棒性，最终将元攻击者的成功率压制至7%，显著提升大模型在开放环境中的安全性与可靠性。代码已公开于https://github.com/sail-sg/LifelongSafetyAlignment。
原文摘要 · Abstract (English)
LLMs have made impressive progress, but their growing capabilities also expose them to highly flexible jailbreaking attacks designed to bypass safety alignment. While many existing defenses focus on known types of attacks, it is more critical to prepare LLMs for unseen attacks that may arise during deployment. To address this, we propose a lifelong safety alignment framework that enables LLMs to continuously adapt to new and evolving jailbreaking strategies. Our framework introduces a competitive setup between two components: a Meta-Attacker, trained to actively discover novel jailbreaking strategies, and a Defender, trained to resist them. To effectively warm up the Meta-Attacker, we first leverage the GPT-4o API to extract key insights from a large collection of jailbreak-related research papers. Through iterative training, the first iteration Meta-Attacker achieves a 73% attack success rate (ASR) on RR and a 57% transfer ASR on LAT using only single-turn attacks. Meanwhile, the Defender progressively improves its robustness and ultimately reduces the Meta-Attacker's success rate to just 7%, enabling safer and more reliable deployment of LLMs in open-ended environments. The code is available at https://github.com/sail-sg/LifelongSafetyAlignment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。