通过监控与干预防止异常代理破坏多智能体协作
Preventing Rogue Agents Improves Multi-Agent Collaboration
- 在动作预测阶段实时监控，提前识别潜在错误行为
- 在多个环境中性能提升最高达20%，有效降低系统失败率
- 适合需要高可靠性的多智能体协同场景
多智能体系统通过分工协作解决共享任务，具备模块化和模拟复杂环境的潜力，但存在单个异常代理即可导致系统崩溃的风险。例如，在知识分布于各代理且通过通信通道共享信息的游戏任务中，任一代理可在不确定时提前终止游戏并做出预测。若能在其行动前检测到此类‘叛逃’行为，可避免系统失败。本文提出在动作预测阶段监测代理行为，并在预判未来错误可能发生时进行干预。为验证方法，我们构建了WhoDunitEnv，支持对任务复杂度和通信结构的模块化控制。在WhoDunitEnv、代码生成任务及资源可持续性模拟环境GovSim上的实验表明，该方法分别带来最高17.4%、2.5%和20%的性能提升。深入分析显示，监控机制能有效识别代理困惑的关键节点，干预措施显著阻止错误传播。
原文摘要 · Abstract (English)
Multi-agent systems, where specialized agents collaborate to solve a shared task hold great potential, from increased modularity to simulating complex environments. However, they also have a major caveat -- a single agent can cause the entire system to fail. Consider a simple game where the knowledge to solve the task is distributed between agents, which share information in a communication channel. At each round, any of the agents can terminate the game and make the final prediction, even if they are uncertain about the outcome of their action. Detection of such rogue agents before they act may prevent the system's failure. In this work, we propose to monitor agents during action prediction and intervene when a future error is likely to occur. To test our approach, we introduce WhoDunitEnv, a multi-agent collaboration environment that allows modular control over task complexity and communication structure. Experiments on WhoDunitEnv, code generation tasks and the GovSim environment for resource sustainability show that our approach leads to substantial performance gains up to 17.4%, 2.5% and 20%, respectively. Thorough analysis shows that our monitors successfully identify critical points of agent confusion and our interventions effectively stop agent errors from propagating.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。