用上下文感知策略管住AI代理，防错防滥用。
AgentGuardian: Learning Access Control Policies to Govern AI Agent Behavior
- 通过监控执行日志学习正常行为，生成自适应访问策略。
- 在真实应用中有效识别恶意输入，误报率低。
- 适合需高安全性的AI自动化系统开发者。
人工智能（AI）代理正被广泛应用于各类场景，以自动化任务、与用户交互并基于数据输入做出决策。确保AI代理仅执行授权操作并恰当处理输入，对维护系统完整性和防止滥用至关重要。本文提出AgentGuardian，一种新型安全框架，通过实施上下文感知的访问控制策略来监管和保护AI代理的运行。在受控预演阶段，该框架监控执行轨迹，学习合法的代理行为和输入模式，并据此推导出调节代理工具调用的自适应策略，策略制定同时考虑实时输入上下文与多步操作间的控制流依赖关系。在两个真实世界的AI代理应用中进行评估表明，AgentGuardian能有效检测恶意或误导性输入，同时保持正常功能。此外，其基于控制流的治理机制可缓解由幻觉引发的错误及其他编排级故障。
原文摘要 · Abstract (English)
Artificial intelligence (AI) agents are increasingly used in a variety of domains to automate tasks, interact with users, and make decisions based on data inputs. Ensuring that AI agents perform only authorized actions and handle inputs appropriately is essential for maintaining system integrity and preventing misuse. In this study, we introduce the AgentGuardian, a novel security framework that governs and protects AI agent operations by enforcing context-aware access-control policies. During a controlled staging phase, the framework monitors execution traces to learn legitimate agent behaviors and input patterns. From this phase, it derives adaptive policies that regulate tool calls made by the agent, guided by both real-time input context and the control flow dependencies of multi-step agent actions. Evaluation across two real-world AI agent applications demonstrates that AgentGuardian effectively detects malicious or misleading inputs while preserving normal agent functionality. Moreover, its control-flow-based governance mechanism mitigates hallucination-driven errors and other orchestration-level malfunctions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。