提出工业调度新基准,对比四种协调模式的优劣。
When Does Hierarchy Help? Benchmarking Agent Coordination in Event-Driven Industrial Scheduling

- 构建分层事件驱动调度环境,模拟真实工业场景。
- 发现集中式高效但难扩展,分层易错,去中心灵活但通信多。
- 适合研究多智能体系统协调机制的学者和工程师。
近期智能体与多智能体系统在工具使用、推理和协作任务中表现优异,但现有评估基准大多局限于弱耦合环境,难以支持对具有层级结构和动态耦合约束的共享演化系统中的协调机制研究。这一空白导致一个关键问题长期未解:不同协调范式在何时成功或失败?为此,我们提出分布式事件驱动调度基准(DESBench),用于评估分层事件驱动调度中的智能体协调能力。该基准基于工业调度中的共享离散事件驱动环境,捕捉多时间尺度决策、部分可观测性及动态耦合约束。我们定义了评估有效性、约束对齐度、协调效率与鲁棒性的任务与指标,并聚焦四种代表性协调范式:集中式、分层式、异层次式和全息式。这些范式对应不同的信息流、决策权与冲突解决机制。受控实验揭示显著协调权衡:集中式鲁棒且通信高效,但随任务难度扩展性差;分层式通过分解提升效率,但存在跨层级偏差;异层次式灵活但通信开销大;全息式约束满足好,却丧失全局鲁棒性。结果表明,协调设计从根本上塑造复杂环境下的智能体系统行为,揭示了仅靠结果指标无法捕捉的结构性权衡,强调未来多智能体系统研究亟需更自适应、有原则、动态的协调机制。
原文摘要 · Abstract (English)
Recent advances in agent and multi-agent systems have shown strong performance on tool use, reasoning, and collaborative tasks. However, existing benchmarks mostly evaluate task completion in weakly coupled environments, and provide limited support for studying coordination in shared, dynamically evolving systems with hierarchy and coupled constraints. This leaves an important question underexplored: when do different coordination paradigms succeed or fail? We introduce Distributed Event-driven Scheduling Benchmark (DESBench), a benchmark for evaluating agent coordination in hierarchical event-driven scheduling. Built on a shared discrete-event driven environment in industrial scheduling, our benchmark captures multi-timescale decision making, partial observability, and dynamically coupled constraints. We define tasks and metrics that evaluate effectiveness, constraint alignment, coordination efficiency, and robustness, and focus on four representative coordination paradigms: centralized, hierarchical, heterarchical, and holonic. These paradigms correspond to distinct mechanisms of information flow, decision authority, and conflict resolution. Our controlled evaluations reveal clear coordination trade-offs: centralized coordination is robust and communication-efficient but scales poorly with difficulty; hierarchical coordination improves efficiency through decomposition but suffers from cross-level misalignment; heterarchical coordination is flexible but communication-heavy; and holonic coordination satisfies constraints well but loses global robustness. These findings demonstrate that coordination design fundamentally shapes agent system behavior in complex environments, revealing structural trade-offs that cannot be captured by outcome metrics alone and underscoring the imperative for more adaptive, principled, and dynamic coordination mechanisms in future MAS research.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。