用内部向量替代文本系统提示，防止提示泄露同时提升模型性能。
You Can't Steal Nothing: Mitigating Prompt Leakages in LLMs via System Vectors
- 将系统提示转为内部向量存储，避免文本形式暴露
- 在GPT-4o、Claude 3.5 Sonnet等模型上有效防住泄露攻击
- 提升指令遵循能力，缓解长上下文遗忘问题
大型语言模型（LLMs）广泛应用于各类场景，依赖定制化系统提示完成任务。尽管开发者通过禁用重复上下文来防范提示泄露，但仍易受新型攻击。本文提出一种简单有效的提示泄露攻击，可从多种LLM应用中提取系统提示，包括GPT-4o、Claude 3.5 Sonnet等前沿模型。由此启发我们探索根本解决方案：彻底移除上下文中的系统提示。为此，我们提出SysVec方法，将系统提示编码为内部表示向量而非原始文本。该方法显著降低未授权披露风险，同时保持模型核心语言能力。实验表明，SysVec有效抵御提示泄露攻击，维护模型功能完整性，并缓解长上下文场景下的遗忘问题。
原文摘要 · Abstract (English)
Large language models (LLMs) have been widely adopted across various applications, leveraging customized system prompts for diverse tasks. Facing potential system prompt leakage risks, model developers have implemented strategies to prevent leakage, primarily by disabling LLMs from repeating their context when encountering known attack patterns. However, it remains vulnerable to new and unforeseen prompt-leaking techniques. In this paper, we first introduce a simple yet effective prompt leaking attack to reveal such risks. Our attack is capable of extracting system prompts from various LLM-based application, even from SOTA LLM models such as GPT-4o or Claude 3.5 Sonnet. Our findings further inspire us to search for a fundamental solution to the problems by having no system prompt in the context. To this end, we propose SysVec, a novel method that encodes system prompts as internal representation vectors rather than raw text. By doing so, SysVec minimizes the risk of unauthorized disclosure while preserving the LLM's core language capabilities. Remarkably, this approach not only enhances security but also improves the model's general instruction-following abilities. Experimental results demonstrate that SysVec effectively mitigates prompt leakage attacks, preserves the LLM's functional integrity, and helps alleviate the forgetting issue in long-context scenarios.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。