保护大模型系统提示，防止被恶意或普通用户泄露
PromptKeeper: Safeguarding System Prompts for LLMs
- 将提示泄露检测转化为假设检验，识别显性和隐性信息外泄
- 发现泄露后用假提示重生成，输出与正常交互无差异
- 既能抵御攻击者提取，又不影响日常对话体验
系统提示广泛用于引导大语言模型（LLMs）的输出，通常包含业务逻辑和敏感信息，其安全性至关重要。然而，攻击性或常规用户查询可能利用LLM漏洞暴露这些隐藏提示。为此，我们提出PromptKeeper，一种防御机制，旨在应对两大核心挑战：可靠检测泄露，以及在泄露发生时缓解侧信道漏洞。通过将检测问题建模为假设检验，PromptKeeper能有效识别显性和细微的泄露行为。一旦检测到泄露，系统会使用一个虚拟提示重新生成响应，确保在无泄露时的输出与正常交互无法区分。PromptKeeper在抵御通过攻击性或常规查询进行的提示提取攻击方面表现稳健，同时在良性用户交互中保持对话能力与运行效率。
原文摘要 · Abstract (English)
System prompts are widely used to guide the outputs of large language models (LLMs). These prompts often contain business logic and sensitive information, making their protection essential. However, adversarial and even regular user queries can exploit LLM vulnerabilities to expose these hidden prompts. To address this issue, we propose PromptKeeper, a defense mechanism designed to safeguard system prompts by tackling two core challenges: reliably detecting leakage and mitigating side-channel vulnerabilities when leakage occurs. By framing detection as a hypothesis-testing problem, PromptKeeper effectively identifies both explicit and subtle leakage. Upon leakage detected, it regenerates responses using a dummy prompt, ensuring that outputs remain indistinguishable from typical interactions when no leakage is present. PromptKeeper ensures robust protection against prompt extraction attacks via either adversarial or regular queries, while preserving conversational capability and runtime efficiency during benign user interactions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。