研究分布式推理中资源竞争的效率损失，提出动态调度优化方案。
The Price of Anarchy in Disaggregated Inference
- 将预填充与解码分离架构建模为多博弈问题，分析资源竞争机制。
- 发现硬件饱和时延迟激增，导致效率损失（PoA）显著上升。
- 设计自适应控制器，实现负载均衡，提升系统吞吐与响应速度。
分布式推理将预填充和解码阶段分离至不同GPU池，形成共享固定算力预算的竞争‘代理’。本文首次对这一架构进行形式化博弈论分析，以NVIDIA Dynamo为案例。将系统建模为三个耦合博弈：预填充与解码池间的双人资源博弈、分层键值缓存上的自私缓存博弈，以及带正外部性的请求路由拥堵博弈。实证验证后两者；前者通过解析方法分析（第9.2节）。揭示GPU饱和引发状态跃迁，改变收益结构：未饱和时，自私行为的价差（PoA）有界；饱和时，超线性延迟与缓存外部性使经验估计的PoA-hat上升。据此设计实时检测饱和的自适应控制器，动态调整路由策略，从缓存偏好转向负载均衡。在3节点NVIDIA B200集群部署Dynamo，运行Nemotron-4-340B（TP=8）与Llama-3.1-70B（TP=4），两模型均呈现三阶段PoA-hat结构，且首个饱和点相同（C=128）。自适应路由使各模型进入更优运行点。最强结果出现在70B单预填充/五解码拓扑，饱和阶段PoA-hat下降3.1倍（66.4降至21.5），吞吐损失仅13%。在70B单预填充/双解码场景，PoA-hat降2.2倍，TTFT P99改善7.6倍（见第8.5节）。
原文摘要 · Abstract (English)
Disaggregated inference architectures physically separate prefill and decode phases onto distinct GPU pools, creating competing "agents" that share a fixed hardware budget. We provide, to our knowledge, the first formal game-theoretic analysis of this architecture, using NVIDIA Dynamo as a concrete case study. We model disaggregated serving as three coupled games: a two-player resource game between prefill and decode pools, a selfish caching game over the hierarchical KV cache, and a congestion game with positive externalities for request routing. We empirically validate the latter two; the P/D resource game is treated analytically (Section 9.2). We characterize how GPU saturation induces regime transitions that shift the game's payoff structure: below saturation, selfish behavior has bounded Price of Anarchy (PoA); at saturation, superlinear latency and cache externalities drive our empirical estimator PoA-hat (defined in Section 6.4) upward. Based on this analysis, we design an adaptive controller that detects saturation transitions in real time and adjusts routing parameters accordingly, shifting from cache-affinity exploitation to load-balanced congestion avoidance. We instantiate our framework on a 3-node NVIDIA B200 cluster running Dynamo with two models, Nemotron-4-340B (TP=8, full-node workers with cross-InfiniBand KV transfers) and Llama-3.1-70B (TP=4), and find the same three-regime PoA-hat structure with the same first post-knee grid point (C=128) on both models. Adaptive routing shifts each model to a better operating point. Our strongest result is on the 70B 1P/5D topology, where PoA-hat drops 3.1x (66.4 to 21.5) in the saturated phase at a 13% throughput cost. On the 70B 1P/2D, PoA-hat drops 2.2x and TTFT P99 drops 7.6x (see Section 8.5).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。