通过优化提示词嵌入，让大模型输出有害内容
DROJ: A Prompt-Driven Attack against Large Language Models
- 在嵌入空间优化攻击提示，引导模型产生有害回应
- 对LLaMA-2-7b-chat实现100%关键词成功率的攻击
- 适合研究模型安全漏洞或对抗攻击的人员阅读
大语言模型（LLMs）在自然语言处理任务中表现出色，但由于训练数据来自互联网，可能生成不当内容。尽管经过大量对齐工作，这些模型仍易受对抗性越狱攻击，即通过特定设计的提示绕过安全机制。本文提出一种新方法——定向表示优化越狱（DROJ），在嵌入层优化越狱提示，使有害查询的隐藏表示向更易引发肯定响应的方向偏移。在LLaMA-2-7b-chat模型上的评估显示，DROJ实现了100%的基于关键词的攻击成功率（ASR），有效规避直接拒绝。但模型偶尔生成重复且无信息量的回复。为此，我们引入一个助益性系统提示，提升输出实用性。代码已公开于https://github.com/Leon-Leyang/LLM-Safeguard。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have demonstrated exceptional capabilities across various natural language processing tasks. Due to their training on internet-sourced datasets, LLMs can sometimes generate objectionable content, necessitating extensive alignment with human feedback to avoid such outputs. Despite massive alignment efforts, LLMs remain susceptible to adversarial jailbreak attacks, which usually are manipulated prompts designed to circumvent safety mechanisms and elicit harmful responses. Here, we introduce a novel approach, Directed Rrepresentation Optimization Jailbreak (DROJ), which optimizes jailbreak prompts at the embedding level to shift the hidden representations of harmful queries towards directions that are more likely to elicit affirmative responses from the model. Our evaluations on LLaMA-2-7b-chat model show that DROJ achieves a 100\% keyword-based Attack Success Rate (ASR), effectively preventing direct refusals. However, the model occasionally produces repetitive and non-informative responses. To mitigate this, we introduce a helpfulness system prompt that enhances the utility of the model's responses. Our code is available at https://github.com/Leon-Leyang/LLM-Safeguard.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。