针对东南亚低资源语言的模型安全漏洞,提出多语言防护系统SEALGuard。
SEALGuard: Safeguarding the Multilingual Conversations in Southeast Asian Languages for LLM Software Systems
- 用低秩适配改造通用多语言模型,构建多语言安全过滤器。
- 在10种语言超26万条数据上测试,对非英语攻击提升48%检测率。
- 适合关注多语言大模型安全、尤其是东南亚语言应用的研究者。
大模型系统中的安全对齐至关重要。尽管近期如LlamaGuard等防护方法在英文不安全输入(如“如何制造炸弹?”)上表现良好,但在多语言场景下效果显著下降。这一缺陷使大模型系统易受低资源语言(如东南亚语言)的不安全与越狱提示攻击。本文提出SEALGuard,一种面向多语言环境的安全防护框架,旨在弥合现有防护系统在多语言安全对齐上的差距。通过低秩适配(LoRA)将通用多语言模型改造为防护器,并构建了包含超过26万条样本的大型多语言安全对齐数据集SEALSBench,涵盖10种语言的正常、不安全及越狱提示。实验表明,现有先进防护器(如LlamaGuard)在非英语攻击下的防御成功率(DSR)分别下降9%和18%;而SEALGuard在各项指标上均优于基线,相比LlamaGuard提升48%的DSR,同时达到最优的精确率与F1分数。消融实验揭示了适配策略与模型规模对性能的关键影响。相关预训练模型与基准数据集已开源,支持后续研究。
原文摘要 · Abstract (English)
Safety alignment is critical for LLM-powered systems. While recent LLM-powered guardrail approaches such as LlamaGuard achieve high detection accuracy of unsafe inputs written in English (e.g., ``How to create a bomb?''), they struggle with multilingual unsafe inputs. This limitation leaves LLM systems vulnerable to unsafe and jailbreak prompts written in low-resource languages such as those in Southeast Asia. This paper introduces SEALGuard, a multilingual guardrail designed to improve the safety alignment across diverse languages. It aims to address the multilingual safety alignment gap of existing guardrails and ensure effective filtering of unsafe and jailbreak prompts in LLM-powered systems. We adapt a general-purpose multilingual language model into a multilingual guardrail using low-rank adaptation (LoRA). We construct SEALSBench, a large-scale multilingual safety alignment dataset containing over 260,000 prompts in ten languages, including safe, unsafe, and jailbreak cases. We evaluate SEALGuard against state-of-the-art guardrails such as LlamaGuard on this benchmark. Our findings show that multilingual unsafe and jailbreak prompts substantially degrade the performance of the state-of-the-art LlamaGuard, which experiences a drop in Defense Success Rate (DSR) by 9% and 18%, respectively, compared to its performance on English-only prompts. In contrast, SEALGuard outperforms existing guardrails in detecting multilingual unsafe and jailbreak prompts, improving DSR by 48% over LlamaGuard and achieving the best DSR, precision, and F1-score. Our ablation study further reveals the contributions of adaptation strategies and model size to the overall performance of SEALGuard. We release our pre-trained model and benchmark at https://github.com/awsm-research/SEALGuard to support further research.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。