为智能体运行时行为建立路径依赖的治理框架，实现安全与效率的动态平衡。
Runtime Governance for AI Agents: Policies on Paths
- 将执行路径作为治理核心，用函数映射身份、路径和动作预测违规概率。
- 证明提示词和静态权限只是该框架的特例，无法处理路径依赖风险。
- 适用于需实时合规的AI应用，如金融、医疗等高风险场景。
AI智能体（基于大语言模型进行规划、推理和行动的系统）表现出非确定性和路径依赖的行为，难以在设计阶段完全实现有效治理——即在尽可能高的任务完成率与法律、数据泄露、声誉损失等成本之间取得平衡。本文提出，执行路径是实现有效运行时治理的核心对象，并将合规策略形式化为确定性函数：输入包括智能体身份、部分执行路径、拟执行动作及组织状态，输出为违规概率。我们指出，提示词（系统提示）和静态访问控制均属于此框架的特例：前者仅影响路径分布而未实际评估路径；后者忽略路径信息，仅针对特定路径子集。我们认为，运行时评估才是通用且必要的方法，尤其适用于路径依赖型策略。本文构建了分析智能体治理的形式化框架，提出受《人工智能法案》启发的具体政策示例，展示参考实现，并识别开放问题，包括风险校准与强制合规的边界。
原文摘要 · Abstract (English)
AI agents -- systems that plan, reason, and act using large language models -- produce non-deterministic, path-dependent behavior that cannot be fully governed at design time, where with governed we mean striking the right balance between as high as possible successful task completion rate and the legal, data-breach, reputational and other costs associated with running agents. We argue that the execution path is the central object for effective runtime governance and formalize compliance policies as deterministic functions mapping agent identity, partial path, proposed next action, and organizational state to a policy violation probability. We show that prompt-level instructions (and "system prompts"), and static access control are special cases of this framework: the former shape the distribution over paths without actually evaluating them; the latter evaluates deterministic policies that ignore the path (i.e., these can only account for a specific subset of all possible paths). In our view, runtime evaluation is the general case, and it is necessary for any path-dependent policy. We develop the formal framework for analyzing AI agent governance, present concrete policy examples (inspired by the AI act), discuss a reference implementation, and identify open problems including risk calibration and the limits of enforced compliance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。