黑盒攻击自动优化提示，让大模型误解用户意图却保持语义相似。
Semantic-Preserving Prompt Hijacking: A Black-Box Adversarial Attack on Auto-Prompt Optimization
- 分层分解输入文本，动态替换关键语言单元。
- 在2400+测试中成功率超越现有方法，保持高语义相似度。
- 适用于检测和防御自动提示优化中的隐蔽攻击。
大型语言模型越来越多地集成自动建议优化模块，在生成最终回复前重写并展示用户输入。尽管该设计旨在提升透明度与可信度，但其自主从多个候选结果中选择单一最优解的过程，使攻击者可通过诱导细微且难以察觉的语义偏移来劫持优化流程。为此，我们提出一种基于黑盒条件的语义保持劫持攻击方法：自适应贪心局部搜索。该方法将输入文本分层分解，掩蔽关键语言单元，并在预设的语义检查点动态调整候选替换词，以最大化模型输出与原始意图的偏差，同时严格保持与原文的语义相似性。在商业及开源大模型上的实验表明，在相同语义相似度约束下，该方法在超过2400个测试案例中实现了高于现有攻击方法的攻击成功率。代码已公开于：https://github.com/franz-chang/DOBS。
原文摘要 · Abstract (English)
LLMs increasingly integrate auto-suggestion optimization modules, enabling them to rewrite and display user input before generating the final response. While this design aims to enhance transparency and trust, its process of autonomously selecting a single best result from multiple candidate solutions allows attackers to hijack this optimization process by inducing subtle, imperceptible semantic shifts. To address this, we propose a semantic preservation hijacking attack method based on black-box conditions: Adaptive Greedy Local Search. This method hierarchically decomposes the input text, masks key language units, and dynamically adjusts candidate replacement words at predefined semantic checkpoints. This maximizes the deviation between the model output and the original intent while strictly maintaining semantic similarity to the original text. Experimental results on commercial and open-source LLMs demonstrate that, under the same semantic similarity constraints, this method achieves a higher attack success rate than existing attack methods in over 2400 test cases. Code is available at: https://github.com/franz-chang/DOBS
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。