用英语语义词典检测多语言越狱攻击,无需训练即可防护
Cross-Lingual Jailbreak Detection via Semantic Codebooks

- 用固定英文词典比对多语言查询的语义相似性,实现免训练防护
- 在标准测试集上准确率接近完美(AUC最高0.99),攻击成功率显著下降
- 适用于多语言安全防护场景,尤其适合不想重新训练的部署者
大语言模型的安全机制仍以英语为主,导致多语言部署存在系统性漏洞。已有研究显示,将恶意提示翻译成其他语言可大幅提升越狱成功率,暴露出跨语言安全缺口。本文探索是否可通过无语言依赖的语义相似性来缓解此类攻击,而无需重训练或语言特异性适配。方法是将多语言查询嵌入与固定英文越狱提示词典进行对比,作为黑盒大模型的免训练外部防护屏障。我们在四种语言、两种翻译管道、四个安全基准、三种嵌入模型和三个目标模型(Qwen、Llama、GPT-3.5)上进行系统评估。结果揭示两种不同的跨语言迁移模式:在包含标准越狱模板的规整基准上,语义相似性能可靠泛化,达到近乎完美的可分性(AUC最高0.99),并在严格低误报率约束下显著降低绝对攻击成功率;但在行为多样且异构的非规整基准上,可分性明显下降(AUC≈0.60–0.70),所有嵌入模型在安全关键的低误报率区域召回率均大幅下降。
原文摘要 · Abstract (English)
Safety mechanisms for large language models (LLMs) remain predominantly English-centric, creating systematic vulnerabilities in multilingual deployment. Prior work shows that translating malicious prompts into other languages can substantially increase jailbreak success rates, exposing a structural cross-lingual security gap. We investigate whether such attacks can be mitigated through language-agnostic semantic similarity without retraining or language-specific adaptation. Our approach compares multilingual query embeddings against a fixed English codebook of jailbreak prompts, operating as a training-free external guardrail for black-box LLMs. We conduct a systematic evaluation across four languages, two translation pipelines, four safety benchmarks, three embedding models, and three target LLMs (Qwen, Llama, GPT-3.5). Our results reveal two distinct regimes of cross-lingual transfer. On curated benchmarks containing canonical jailbreak templates, semantic similarity generalizes reliably across languages, achieving near-perfect separability (AUC up to 0.99) and substantial reductions in absolute attack success rates under strict low-false-positive constraints. However, under distribution shift - on behaviorally diverse and heterogeneous unsafe benchmarks - separability degrades markedly (AUC $\approx$ 0.60-0.70), and recall in the security-critical low-FPR regime drops across all embedding models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。