用智能体团队自动探测大模型提示词泄露漏洞。
Automating Prompt Leakage Attacks on Large Language Models Using Agentic Approach
- 设计多智能体系统，主动探测目标模型的敏感提示信息。
- 提出可区分性测试标准，确保攻击者无法识别原始与脱敏提示。
- 为模型安全评估提供可复现的自动化对抗测试方法。
本文提出一种新型方法，用于评估大语言模型（LLMs）在提示词泄露方面的安全性——即系统级提示或专有配置的暴露风险。定义提示词泄露为部署安全的关键威胁，引入基于智能体团队的测试框架。利用AG2（前身为AutoGen），构建多智能体系统，协作探查并尝试从目标模型中提取其内部提示。借鉴密码学中的安全定义，提出“提示词泄露安全”系统：攻击者无法区分初始化为原始提示的智能体与仅含非敏感信息提示的智能体。在安全系统中，两者的输出对攻击者不可区分，从而保障敏感信息不外泄。该基于密码学思想的框架为大模型安全评估与设计提供了严谨标准。本研究建立了一套系统性的对抗测试方法，弥合了自动化威胁建模与实际模型安全之间的鸿沟。代码已开源于GitHub。
原文摘要 · Abstract (English)
This paper presents a novel approach to evaluating the security of large language models (LLMs) against prompt leakage-the exposure of system-level prompts or proprietary configurations. We define prompt leakage as a critical threat to secure LLM deployment and introduce a framework for testing the robustness of LLMs using agentic teams. Leveraging AG2 (formerly AutoGen), we implement a multi-agent system where cooperative agents are tasked with probing and exploiting the target LLM to elicit its prompt. Guided by traditional definitions of security in cryptography, we further define a prompt leakage-safe system as one in which an attacker cannot distinguish between two agents: one initialized with an original prompt and the other with a prompt stripped of all sensitive information. In a safe system, the agents' outputs will be indistinguishable to the attacker, ensuring that sensitive information remains secure. This cryptographically inspired framework provides a rigorous standard for evaluating and designing secure LLMs. This work establishes a systematic methodology for adversarial testing of prompt leakage, bridging the gap between automated threat modeling and practical LLM security. You can find the implementation of our prompt leakage probing on GitHub.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。