让智能体组合更安全,能自动避开高风险失败路径。
Risk-Sensitive Agent Compositions
- 用有向无环图建模智能体组合,通过动态规划找最优路径。
- 首次在组合中最小化极端风险(如安全违规),提升可靠性。
- 适合机器人控制、AI系统集成等对安全性要求高的场景。
现代智能体系统将复杂任务分解为子任务,并选择专用AI智能体完成。本文将智能体工作流形式化为有向无环图(agent graphs),其中边代表智能体,路径对应可行的组合。真实部署需在最大化任务成功率的同时,最小化安全、公平、隐私等要求的违反,这依赖于对组合低概率(尾部)行为的精细分析。本文研究在可行组合中最小化损失分布的条件风险价值(CVaR)与风险价值(VaR),损失量化各类违规程度。提出一种高效算法,通过动态规划结合并集界近似VaR,证明其在一大类实际损失函数下渐近近似最优。同时可作为副产品估算CVaR。在多个类似视频游戏的强化学习控制基准上评估,验证了该算法在逼近VaR和识别最优组合方面的有效性。
原文摘要 · Abstract (English)
From software development to robot control, modern agentic systems decompose complex objectives into a sequence of subtasks and choose a set of specialized AI agents to complete them. We formalize agentic workflows as directed acyclic graphs, called agent graphs, where edges represent AI agents and paths correspond to feasible compositions of agents. Real-world deployment requires selecting agent compositions that not only maximize task success but also minimize violations of safety, fairness, and privacy requirements which demands a careful analysis of the low-probability (tail) behaviors of compositions of agents. In this work, we consider risk minimization over the set of feasible agent compositions and seek to minimize the value-at-risk and the conditional value-at-risk of the loss distribution of the agent composition where the loss quantifies violations of these requirements. We introduce an efficient algorithm which traverses the agent graph and finds a near-optimal composition of agents. It uses a dynamic programming approach to approximate the value-at-risk of agent compositions by exploiting a union bound. Furthermore, we prove that the approximation is near-optimal asymptotically for a broad class of practical loss functions. We also show how our algorithm can be used to approximate the conditional value-at-risk as a byproduct. To evaluate our framework, we consider a suite of video game-like control benchmarks that require composing several agents trained with reinforcement learning and demonstrate our algorithm's effectiveness in approximating the value-at-risk and identifying the optimal agent composition.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。