攻击者通过微调嵌入层绕过安全对齐，让大模型输出有害内容。
Embedding Poisoning: Bypassing Safety Alignment via Embedding Semantic Shift
- 在嵌入层注入微小扰动，不改权重也不改输入文本
- 六款对齐模型平均96.43%成功率，仍保持正常任务性能
- 可规避传统检测，适合研究模型安全的开发者参考
大型语言模型通过Hugging Face等平台广泛分发，带来显著安全挑战。尽管这些平台进行基础安全扫描，却常无法检测嵌入层中的细微篡改。本文揭示一种新型部署阶段攻击，通过向嵌入层输出注入难以察觉的扰动（不修改模型权重或输入文本），虽统计上无异常，却能系统性绕过安全对齐机制，诱导推理时产生有害行为。我们提出搜索式嵌入投毒（SEP），一种无需模型知识、可泛化的框架，针对高风险词元的嵌入引入精心优化的扰动。SEP利用模型响应从拒绝到有害输出再到语义偏离的可预测线性变化，定位出能避开对齐防护的窄扰动窗口。在六款对齐的大模型上评估，平均攻击成功率达96.43%，同时保持良性任务性能，并逃逸常规检测机制。研究揭示了部署安全中的关键疏漏，强调未来需加强嵌入层完整性校验。
原文摘要 · Abstract (English)
The widespread distribution of Large Language Models (LLMs) through public platforms like Hugging Face introduces significant security challenges. While these platforms perform basic security scans, they often fail to detect subtle manipulations within the embedding layer. This work identifies a novel class of deployment phase attacks that exploit this vulnerability by injecting imperceptible perturbations directly into the embedding layer outputs without modifying model weights or input text. These perturbations, though statistically benign, systematically bypass safety alignment mechanisms and induce harmful behaviors during inference. We propose Search based Embedding Poisoning(SEP), a practical, model agnostic framework that introduces carefully optimized perturbations into embeddings associated with high risk tokens. SEP leverages a predictable linear transition in model responses, from refusal to harmful output to semantic deviation to identify a narrow perturbation window that evades alignment safeguards. Evaluated across six aligned LLMs, SEP achieves an average attack success rate of 96.43% while preserving benign task performance and evading conventional detection mechanisms. Our findings reveal a critical oversight in deployment security and emphasize the urgent need for embedding level integrity checks in future LLM defense strategies.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。