用对抗自博弈让模型在各种人设下都安全，防住新型越狱攻击。
Disentangling Intent from Role: Adversarial Self-Play for Persona-Invariant Safety Alignment
- 通过攻击方的谱系演化和防御方的一致性学习实现双向进化
- 防御后攻击成功率大幅下降，同时保持模型通用能力
- 适合关注大模型安全对齐与对抗训练的研究者
大型语言模型（LLMs）能力日益增强，已广泛应用于多个领域，包括高风险场景。尽管安全对齐技术有所进展，当前模型仍易受基于人设的越狱攻击影响。现有研究多聚焦于攻击迭代，缺乏系统性的防御机制。为此，我们提出人格不变对齐（PIA），一种对抗自博弈框架，攻击侧采用人格谱系演化（PLE），防御侧采用人格不变一致性学习（PICL）。理论上，PICL基于结构解耦假设，通过单向KL散度约束实现安全决策与人格上下文的解耦，确保在人格越狱攻击下仍保持安全行为。实验表明，PLE通过谱系信用传播高效探索高风险人格空间；而PICL显著降低攻击成功率（ASR），同时保留模型通用能力，验证了该对齐范式的优越性与鲁棒性。代码已开源：https://github.com/JiajiaLi-1130/PIA。
原文摘要 · Abstract (English)
The growing capabilities of large language models (LLMs) have driven their widespread deployment across diverse domains, even in potentially high-risk scenarios. Despite advances in safety alignment techniques, current models remain vulnerable to emerging persona-based jailbreak attacks. Existing research on persona-based jailbreak has primarily focused on attack iterations, yet it lacks systemic and mechanistic constraints on the defense side. To address this challenge, we propose Persona-Invariant Alignment (PIA), an adversarial self-play framework that achieves co-evolution through Persona Lineage Evolution (PLE) on the attack side and Persona-Invariant Consistency Learning (PICL) on the defense side. Theoretically, PICL is grounded in the structural separation hypothesis, using a unilateral KL-divergence constraint to enable the structural decoupling of safety decisions from persona context, thereby maintaining safe behavior under persona-based jailbreak attacks. Experimental results demonstrate that PLE efficiently explores high-risk persona spaces by leveraging lineage-based credit propagation. Meanwhile, the PICL defense method significantly reduces the Attack Success Rate (ASR) while preserving the model's general capability, thereby validating the superiority and robustness of this alignment paradigm. Codes are available at https://github.com/JiajiaLi-1130/PIA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。