让多智能体系统实时感知硬件状态,自动调优性能与延迟。
INFRAMIND: Infrastructure-Aware Multi-Agent Orchestration

- 根据实时资源负载动态调整任务架构和模型选择
- 高负载下延迟降低7倍,99.9%服务承诺达标
- 适合需要稳定响应的生产级多模型应用
现有大模型多智能体编排方法忽略运行时基础设施状态。在共享GPU集群高并发场景下,这种盲区导致资源严重浪费:理想模型排队积压,同等能力的替代模型却闲置。在多步骤串行调用的管道中,延迟逐层累积。解决此问题需实时处理队列深度、缓存压力、延迟等动态噪声信号,并做出规划、路由与调度三重决策。我们提出INFRAMIND框架,使整个多智能体栈具备基础设施感知能力。其规划器依据实时负载与预算,拥堵时倾向简化拓扑,低负载时启用复杂结构;执行器在每步观测各模型队列、缓存使用率和响应延迟,决定调用模型及推理深度;调度器则按优先级重排请求队列,确保紧急任务优先。该系统以分层约束马尔可夫决策过程建模,通过强化学习端到端训练,自动平衡质量与延迟。在五个基准测试中,低负载下准确率最高提升7.6个百分点,延迟降低7倍;高负载下保持99.9%的服务等级协议合规,而所有基线均低于50%。
原文摘要 · Abstract (English)
Existing multi-agent LLM orchestration methods, ranging from brute-force ensembles to learned routers, select models and topologies based on task and model features. However, these methods do not consider the runtime state of the serving infrastructure. On shared GPU clusters under concurrent load, this infrastructure blindness causes systematic resource underutilization: preferred models accumulate deep request queues while equally capable alternatives sit idle. In multi-agent pipelines, where each query triggers multiple sequential model calls, these delays then compound across every downstream step. Closing this gap is challenging because the relevant infrastructure signals (queue depths, KV-cache pressure, latencies) are dynamic and noisy, and they must drive three different decisions: planning, per-step routing, and scheduling. We introduce INFRAMIND, a framework that makes the entire multi-agent stack infrastructure-aware. An infra-aware planner conditions topology and role selection on real-time system load and remaining budget, biasing toward simpler graphs under congestion and richer ones at low load. An infra-aware executor then observes per-model queue depths, cache utilization, and response latencies at each agent step to decide which model to call and how deeply to reason; a budget-aware scheduler further reorders each model's queue so that urgent requests are served first. Cast as a hierarchical constrained MDP and solved end-to-end via reinforcement learning, the system learns to balance quality against latency automatically. Across five benchmarks, INFRAMIND delivers up to +7.6 pp accuracy over the prior baseline at low load with up to 7x lower latency, and sustains up to 99.9% SLO compliance under high load where every baseline drops below 50%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。