发现文本嵌入模型偏移问题，用通用魔法词攻破大模型安全防线
Jailbreaking LLMs' Safeguard with Universal Magic Words for Text Embedding Models
- 通过寻找通用魔法词，操纵文本嵌入方向实现攻击
- 在JailbreakBench上显著降低防护效果，跨模型跨语言泛化
- 适合安全研究者、防御设计者关注，警示嵌入层风险
大型语言模型的安全问题近来受到广泛关注，基于文本嵌入模型的防护机制成为基础防线。测试发现，文本嵌入模型输出分布存在严重偏移，均值较大。基于此，我们提出高效方法搜索通用魔法词，作为后缀可将任意文本嵌入向量推向偏移方向，从而改变任意文本对的相似度，误导安全防护机制。攻击者可通过在用户提示后添加魔法词，并要求LLM以魔法词结尾，实现越狱。实验表明，该攻击在JailbreakBench上显著削弱防护性能，真实聊天机器人在全链路攻击下产生有害输出，且具备跨输入/输出文本、模型与语言的泛化能力。为消除此风险，我们还提出无需训练即可纠正嵌入偏移并提升下游性能的防御方法。
原文摘要 · Abstract (English)
The security issue of large language models (LLMs) has gained wide attention recently, with various defense mechanisms developed to prevent harmful output, among which safeguards based on text embedding models serve as a fundamental defense. Through testing, we discover that the output distribution of text embedding models is severely biased with a large mean. Inspired by this observation, we propose novel, efficient methods to search for **universal magic words** that attack text embedding models. Universal magic words as suffixes can shift the embedding of any text towards the bias direction, thus manipulating the similarity of any text pair and misleading safeguards. Attackers can jailbreak the safeguards by appending magic words to user prompts and requiring LLMs to end answers with magic words. Experiments show that magic word attacks significantly degrade safeguard performance on JailbreakBench, cause real-world chatbots to produce harmful outputs in full-pipeline attacks, and generalize across input/output texts, models, and languages. To eradicate this security risk, we also propose defense methods against such attacks, which can correct the bias of text embeddings and improve downstream performance in a train-free manner.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。