防止大模型代理权限提升攻击，保障运行时安全
Prompt Flow Integrity to Prevent Privilege Escalation in LLM Agents
- 通过代理隔离、安全数据处理和权限防护三策略防御攻击
- 实测有效阻止权限提升，同时保持代理功能可用
- 适合关注大模型应用安全的开发者与研究人员
大语言模型（LLMs）与工具结合形成强大智能代理，提供广泛服务。与传统软件不同，其行为由用户或工具数据在运行时生成的自然语言提示决定，具备高度灵活性和可编程性，但也带来新安全风险，易受权限提升攻击。用户提示可能被代理以不安全方式解析，引发非确定性行为，被攻击者利用。为此，我们提出面向系统安全的提示流完整性（PFI）机制，基于代理架构特性，采用代理隔离、安全不可信数据处理及权限提升防护三重技术。评估结果表明，PFI能有效缓解权限提升攻击，同时维持代理的实用性。
原文摘要 · Abstract (English)
Large Language Models (LLMs) are combined with tools to create powerful LLM agents that provide a wide range of services. Unlike traditional software, LLM agent's behavior is determined at runtime by natural language prompts from either user or tool's data. This flexibility enables a new computing paradigm with unlimited capabilities and programmability, but also introduces new security risks, vulnerable to privilege escalation attacks. Moreover, user prompts are prone to be interpreted in an insecure way by LLM agents, creating non-deterministic behaviors that can be exploited by attackers. To address these security risks, we propose Prompt Flow Integrity (PFI), a system security-oriented solution to prevent privilege escalation in LLM agents. Analyzing the architectural characteristics of LLM agents, PFI features three mitigation techniques -- i.e., agent isolation, secure untrusted data processing, and privilege escalation guardrails. Our evaluation result shows that PFI effectively mitigates privilege escalation attacks while successfully preserving the utility of LLM agents.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。