arXiv:2605.01280cs.DCcs.AI2026-05被引 1

LLM推理服务需数学优化,而非依赖随意的调度策略。

Position: LLM Serving Needs Mathematical Optimization and Algorithmic Foundations, Not Just Heuristics

论文配图:Position: LLM Serving Needs Mathematical Optimization and Algorithmic Foundations, Not Just Heuristics
图 1 · 摘自论文原文
  • 用数学模型刻画LLM推理的动态特性,设计可证明性能的算法。
  • 现有调度策略如FIFO、LRU在复杂场景下表现不可靠。
  • 适合关注系统优化与理论保证的研究者阅读。

本文主张,大语言模型推理服务已超越通用启发式方法,亟需数学优化与算法基础。尽管vLLM和SGLang等系统进展迅速,其核心算法仍沿用经典分布式计算范式:请求路由采用最短队列优先或轮询,调度默认为先进先出(FIFO),KV缓存淘汰遵循最近最少使用(LRU)。这些通用策略忽视了LLM推理的独特结构——动态增长的KV缓存内存、预填充-解码阶段不对称性、输出长度未知以及持续批处理约束。我们主张建立能捕捉这些特性的数学模型,以设计在多样工作负载下具备可证明性能保障的算法,而非依赖可能在某些场景有效但在其他场景失效的启发式方法。新兴研究显示,运筹学与机器学习系统交叉领域的严谨方法可达到甚至超越启发式性能,并提供理论保证。我们呼吁社区将LLM服务算法设计视为前沿研究领域。

原文摘要 · Abstract (English)

This position paper argues that LLM inference serving has outgrown generic heuristics and now demands mathematical optimization and algorithmic foundations. Despite rapid advances in serving systems such as vLLM and SGLang, their algorithmic cores remain largely unchanged from classical distributed computing: request routing uses join-shortest-queue or round-robin, scheduling defaults to FIFO, and KV cache eviction follows LRU. These general-purpose policies ignore the distinctive structure of LLM inference--dynamically growing KV cache memory, prefill-decode phase asymmetry, unknown output lengths, and continuous batching constraints. We contend that the field must develop mathematical models capturing these characteristics, enabling the design of algorithms with provable performance guarantees across diverse workloads, rather than heuristics that may succeed in some scenarios but fail unpredictably in others. Emerging work at the intersection of operations research and ML systems demonstrates that principled methods can match or exceed heuristic performance while providing theoretical guarantees. We call on the community to recognize algorithmic design for LLM serving as a research frontier.

LLM服务算法优化数学建模

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。