用自蒸馏让大模型跨语言传递安全能力,低成本解决低资源语言漏洞
Multilingual Safety Alignment via Self-Distillation

- 通过自蒸馏将高资源语言的安全能力迁移到低资源语言
- 仅需多语言查询即可实现跨语言安全对齐,无需人工标注响应数据
- 新提出双视角权重机制,动态强化关键安全词的惩罚
大型语言模型在多语言场景中存在严重安全错位:高资源语言(如英语)有强防护,而低资源语言(如爪哇语)极易受越狱攻击。现有对齐方法依赖每种语言的高质量响应数据,成本高昂。本文提出跨语言安全迁移框架Multilingual Self-Distillation(MSD),将模型在高资源语言中的内在安全能力迁移至低资源语言,无需任何目标语言的响应数据。框架灵活,可结合多种自蒸馏策略,我们实现了两种具体方法——同策略与异策略自蒸馏,均仅使用多语言查询完成跨语言安全转移。此外,提出双视角安全加权(DPSW),从教师与学生双角度优化蒸馏目标,动态提升安全关键词的惩罚权重,降低非关键词权重。在多个代表性模型和多语言越狱/效用基准上实验表明,该方法持续优于基线,能有效泛化至更难数据集和未见语言,同时保持模型通用能力。
原文摘要 · Abstract (English)
Large language models (LLMs) exhibit severe multilingual safety misalignment: they possess strong safeguards in high-resource languages but remain highly vulnerable to jailbreak attacks in low-resource languages. Current safety alignment methods generally rely on high-quality response data for each target language, which is expensive and difficult to generate. In this paper, we propose a cross-lingual safeguard transfer framework named Multilingual Self-Distillation (MSD). This framework transfers an LLM's inherent safety capabilities from high-resource (e.g., English) to low-resource (e.g., Javanese) languages, overcoming the need for response data in any language. Our framework is flexible and can be integrated with different self-distillation strategies. Specifically, we implement two concrete methods -- on-policy MSD and off-policy MSD -- both of which enable effective cross-lingual safety transfer using only multilingual queries. Furthermore, we propose Dual-Perspective Safety Weighting (DPSW), a divergence measure to optimize the distillation objective. By jointly considering the perspectives of both the teacher and the student, DPSW adaptively increases the penalty weights on safety-critical tokens while reducing the weights on non-critical tokens. Extensive experiments on representative LLMs across diverse multilingual jailbreak and utility benchmarks demonstrate that our method consistently achieves superior multilingual safety performance. Notably, it generalizes effectively to more challenging datasets and unseen languages while preserving the model's general capabilities.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。