用稀疏自编码器定位关键特征,精准生成能绕过防御的恶意文本。
Layer-Wise Perturbations via Sparse Autoencoders for Adversarial Text Generation
- 通过稀疏自编码器识别文本中高激活的关键语义特征。
- 对特定层特征进行选择性扰动,生成可绕过主流防御的对抗文本。
- 为红队测试提供兼顾攻击效果与安全对齐的新方法,适合模型安全研究者。
随着自然语言处理,尤其是大语言模型(LLMs)的快速发展,生成能够绕过模型限制的对抗样本仍是理解模型漏洞和提升鲁棒性的关键挑战。本文提出一种新型黑盒攻击方法,利用大模型的可解释性。我们引入稀疏特征扰动框架(SFPF),通过稀疏自编码器(SAE)识别并操纵文本中的关键特征。在使用SAE重建隐藏层表征后,对成功攻击的文本进行特征聚类,识别出激活值较高的特征,并对其进行扰动以生成新的对抗文本。该选择性扰动在保持恶意意图的同时增强安全信号,从而提高其规避现有防御机制的能力。实验表明,SFPF生成的对抗文本可绕过当前最先进的防御机制,揭示了现有NLP系统存在的持续性漏洞。然而,该方法在不同提示和层间的有效性存在差异,其在其他架构及更大模型上的泛化能力仍有待验证。
原文摘要 · Abstract (English)
With the rapid proliferation of Natural Language Processing (NLP), especially Large Language Models (LLMs), generating adversarial examples to jailbreak LLMs remains a key challenge for understanding model vulnerabilities and improving robustness. In this context, we propose a new black-box attack method that leverages the interpretability of large models. We introduce the Sparse Feature Perturbation Framework (SFPF), a novel approach for adversarial text generation that utilizes sparse autoencoders to identify and manipulate critical features in text. After using the SAE model to reconstruct hidden layer representations, we perform feature clustering on the successfully attacked texts to identify features with higher activations. These highly activated features are then perturbed to generate new adversarial texts. This selective perturbation preserves the malicious intent while amplifying safety signals, thereby increasing their potential to evade existing defenses. Our method enables a new red-teaming strategy that balances adversarial effectiveness with safety alignment. Experimental results demonstrate that adversarial texts generated by SFPF can bypass state-of-the-art defense mechanisms, revealing persistent vulnerabilities in current NLP systems.However, the method's effectiveness varies across prompts and layers, and its generalizability to other architectures and larger models remains to be validated.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。