arXiv:2601.21351cs.LGcs.AI2026-01

提出可解析的资源分配方法,优化大模型解码中注意力与前馈网络的配置比例。

Analytical Provisioning for Attention-FFN Disaggregated LLM Serving under Stochastic Workloads

  • 基于稳态负载建模,用单一统计量θ表征随机工作负载
  • 推导出最优注意力/前馈比的闭式解,涵盖通信与计算瓶颈
  • 适用于不同提示和生成长度分布,适合生产环境部署

注意力-前馈网络解耦(AFD)是一种新兴的大语言模型解码架构,将依赖状态的注意力计算与无状态的前馈网络计算分离,通过每步通信连接。尽管可独立扩展内存与计算资源,其性能对注意力/前馈配置比例极为敏感:配置不当会导致步骤级阻塞和设备空闲浪费。本文针对随机工作负载下的rA–1F拓扑结构,构建了AFD集群的解析资源配置框架。两个随机性来源构成核心挑战:每槽注意力负载随KV缓存增长及请求的随机提示与生成长度动态变化;多注意力节点同步执行引入由最慢节点决定的屏障。我们通过更新-奖励方法刻画每槽稳态令牌负载,识别出一个决定资源配置的关键统计量θ,该量可从请求日志非参数估计。分析得到一个闭式均场规则,明确区分注意力、通信与前馈瓶颈三类情形,并进一步提出高斯屏障修正以量化跨节点同步开销。基于真实轨迹校准的AFD模拟器验证了框架有效性:预测最优比与仿真最优比误差小于10%。整体成果提供了一套紧凑、可校准的理论工具,揭示随机工作负载结构如何决定分布式大模型服务的资源配置。

原文摘要 · Abstract (English)

Attentio-FFN disaggregation (AFD) is an emerging architecture for LLM decoding that separates state-heavy, KV-cache-dominated Attention computation from stateless, compute-intensive FFN computation, connected by per-step communication. While AFD enables independent scaling of memory and compute resources, its performance is highly sensitive to the Attention/FFN provisioning ratio: mis-sizing induces step-level blocking and costly device idle time. We develop an analytical provisioning framework for AFD bundles in an $r$A--$1$F topology under stochastic workloads. Two sources of randomness shape the problem: per-slot Attention workload evolves as KV caches grow and completed requests are replenished with random prompt and decode lengths, and synchronized execution across Attention workers introduces a barrier governed by the slowest worker. We address both via a renewal-reward characterization of the per-slot stationary token load, identifying a single workload statistic $θ$ that governs provisioning under arbitrary prefill-decode distributions and admits a nonparametric estimator from request traces. The analysis yields a closed-form mean-field rule for the optimal A/F ratio decomposing into Attention-, communication-, and FFN-bottleneck regimes, together with a Gaussian barrier-aware refinement that quantifies cross-worker synchronization overhead. A trace-calibrated AFD simulator supports the framework across workloads: the predicted optimal ratio matches the simulation-optimal within 10%. Together, these results provide a compact, calibratable account of how stochastic workload structure determines provisioning in disaggregated LLM serving.

大模型推理资源调度系统优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。