arXiv:2504.11703cs.CRcs.AI2025-04被引 102

用权限控制保护AI代理,防攻击且不降低效率

Progent: Securing AI Agents with Privilege Control

  • 将权限设为符号规则,自动判断工具调用是否合法
  • 攻击成功率大幅下降,任务完成率仍保持高位
  • 适合需安全可控的AI代理部署场景

AI代理通过工具调用与外部环境交互,易受间接提示注入等攻击,引发未授权操作。现有安全机制面临自主性、概率性行为及安全与效用间权衡等挑战。本文提出Progent框架,通过权限控制保障代理安全:将权限定义为基于工具名和参数的符号规则,规定哪些调用可执行、哪些应被阻止。所有工具调用均经确定性检查,遵循最小权限原则。针对多样任务与动态执行状态,利用大模型从用户任务生成初始策略,并在执行中根据新信息自动更新。每次更新由SMT求解器判定为缩小(自动应用)或扩展(需显式批准),确保动作空间仅能缩小而不可越权,实现单调约束。该机制在对抗输入下仍能防止权限静默提升,保持高实用性。在AgentDojo与ASB等主流基准上评估显示,Progent显著降低攻击成功率,同时维持高任务完成率。进一步在LangChain与OpenAI Agents SDK中验证了其实际可用性。

原文摘要 · Abstract (English)

AI agents interact with external environments through tool calls, exposing them to attacks like indirect prompt injection that can trigger unauthorized actions. Securing these agents is challenging: they behave autonomously and probabilistically, security requirements evolve depending on the user's task and execution state, and there is an inherent tradeofff between security and utility. In this work, we introduce Progent, a novel framework that secures AI agents via privilege control. Progent represents privilege as a security policy consisting of symbolic rules over tool names and arguments. These rules specify which tool calls are allowed for task completion and which unnecessary ones are blocked for security. Every tool call is checked against such a policy through a deterministic procedure, enforcing the principle of least privilege. To handle diverse user tasks and evolving execution contexts, an LLM automatically generates the initial policy from the user's task and updates it during execution as new information arrives. Each proposed update is determined by an SMT solver to be either a narrowing (applied automatically) or an expansion (requiring explicit approval), ensuring that the agent's effective action space can only shrink without approval (monotonic confinement). This deterministic update mechanism preserves utility and prevents silent privilege escalation, even when adversarial inputs are present. Our evaluation on popular benchmarks (i.e., AgentDojo and ASB) shows that Progent significantly reduces attack success rates while maintaining high utility. We further validate Progent's practicality by showcasing its effectiveness in real-world agent frameworks such as LangChain and OpenAI Agents SDK.

AI安全权限控制代理系统

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。