用意图伪装攻击让大模型误判恶意请求为正常提问。
Friend or Foe: How LLMs' Safety Mind Gets Fooled by Intent Shift Attack
- 通过变换请求意图生成看似无害的自然语言攻击
- 相比直接攻击成功率提升70%以上,微调后接近100%
- 揭示模型对意图理解的脆弱性,适合安全研究者参考
大型语言模型(LLMs)尽管能力强大,仍易受越狱攻击。现有攻击多通过添加干扰上下文或对抗性标记来迷惑模型,但核心恶意意图不变。本文提出ISA(意图转移攻击),通过构建意图转换分类体系,生成可被模型误判为良性信息请求的攻击。与依赖复杂标记或长上下文的方法不同,ISA仅需对原始请求进行最小修改,即可生成自然、可读性强且看似无害的提示。在开源和商用模型上的大量实验表明,ISA相较直接恶意提示的攻击成功率提升超70%;仅用经ISA模板重构的良性数据微调模型,成功率即逼近100%。防御方面,我们评估现有方法均无法有效应对ISA,同时探索了无需训练和基于训练的缓解策略。研究揭示了大模型在意图识别上的根本缺陷,凸显亟需更有效的安全防护机制。代码与数据集已公开于https://github.com/NJUNLP/ISA。
原文摘要 · Abstract (English)
Large language models (LLMs) remain vulnerable to jailbreaking attacks despite their impressive capabilities. Investigating these weaknesses is crucial for robust safety mechanisms. Existing attacks primarily distract LLMs by introducing additional context or adversarial tokens, leaving the core harmful intent unchanged. In this paper, we introduce ISA (Intent Shift Attack), which obfuscates LLMs about the intent of the attacks. More specifically, we establish a taxonomy of intent transformations and leverage them to generate attacks that may be misperceived by LLMs as benign requests for information. Unlike prior methods relying on complex tokens or lengthy context, our approach only needs minimal edits to the original request, and yields natural, human-readable, and seemingly harmless prompts. Extensive experiments on both open-source and commercial LLMs show that ISA achieves over 70% improvement in attack success rate compared to direct harmful prompts. More critically, fine-tuning models on only benign data reformulated with ISA templates elevates success rates to nearly 100%. For defense, we evaluate existing methods and demonstrate their inadequacy against ISA, while exploring both training-free and training-based mitigation strategies. Our findings reveal fundamental challenges in intent inference for LLMs safety and underscore the need for more effective defenses. Our code and datasets are available at https://github.com/NJUNLP/ISA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。