用少量高资源语言训练，让安全模型覆盖100种语言。
CREST: Universal Safety Guardrails Through Cluster-Guided Cross-Lingual Transfer
- 从13种高资源语言聚类迁移，实现跨语言安全分类。
- 仅0.5B参数，在6个基准上超越同类模型表现。
- 适合需要低成本覆盖多语言安全的落地应用。
确保大语言模型的内容安全对实际应用至关重要。然而现有安全防护系统主要针对高资源语言，导致大量低资源语言使用者被忽视。为此，我们提出CREST（CRoss-lingual Efficient Safety Transfer），一种参数高效、支持100种语言的安全分类模型，仅需0.5B参数。通过在精选的13种高资源语言上训练，利用基于聚类的跨语言迁移，实现从少数语言向100种语言的有效泛化，涵盖未见的高资源与低资源语言。该方法缓解了低资源语言数据不足的问题。我们在六个安全基准上进行综合评估，结果表明，CREST在同等规模下优于现有最先进防护系统，并在性能上媲美参数量达2.5B及以上的模型。研究揭示了语言特异性防护系统的局限性，强调构建可扩展的通用、无语言依赖安全系统的重要性，以服务全球用户。
原文摘要 · Abstract (English)
Ensuring content safety in large language models (LLMs) is essential for their deployment in real-world applications. However, existing safety guardrails are predominantly tailored for high-resource languages, leaving a significant portion of the world's population underrepresented who communicate in low-resource languages. To address this, we introduce CREST (CRoss-lingual Efficient Safety Transfer), a parameter-efficient multilingual safety classification model that supports 100 languages with only 0.5B parameters. By training on a strategically chosen subset of only 13 high-resource languages, our model utilizes cluster-based cross-lingual transfer from a few to 100 languages, enabling effective generalization to both unseen high-resource and low-resource languages. This approach addresses the challenge of limited training data in low-resource settings. We conduct comprehensive evaluations across six safety benchmarks to demonstrate that CREST outperforms existing state-of-the-art guardrails of comparable scale and achieves competitive results against models with significantly larger parameter counts (2.5B parameters and above). Our findings highlight the limitations of language-specific guardrails and underscore the importance of developing universal, language-agnostic safety systems that can scale effectively to serve global populations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。