arXiv:2605.21312cs.DCcs.AI2026-05被引 2

构建了高精度的LLM推理仿真系统,可模拟复杂部署架构与工作负载。

Frontier: Towards Comprehensive and Accurate LLM Inference Simulation

论文配图:Frontier: Towards Comprehensive and Accurate LLM Inference Simulation
图 1 · 摘自论文原文
  • 采用解耦抽象建模,支持预填充-解码、注意力-前馈解耦等现代部署结构
  • 在16张H800 GPU上平均吞吐误差低于4%,端到端延迟误差降至2.6%
  • 适合系统设计者验证智能体推理调度、强化学习后训练重构等新场景

现代大模型服务已不再单一或同质化,而是融合解耦执行、复杂并行、运行时优化以及有状态任务(如推理、智能体、强化学习推演)。仿真有助于探索这一日益复杂的架构空间,但现有仿真器缺乏应有的架构完整性与决策级保真度。其单体复制抽象不适用于解耦部署,平均情况分析代理会扭曲服务等级协议(SLA)预测,甚至颠倒优化结论。本文提出Frontier,一个面向现代大模型推理服务的离散事件仿真器。Frontier采用解耦抽象,通过角色化集群工作节点建模共置、预填充-解码解耦(PDD)、注意力-前馈解耦(AFD),将关键运行时优化(如CUDA Graphs、推测性解码)嵌入调度-批处理引擎循环,并支持新兴有状态请求。它能准确预测多种服务场景下的计算、通信和内存开销。在16台H800 GPU测试平台上,Frontier平均吞吐误差低于4%;相比最先进仿真器,在共置下端到端延迟误差由44.9%降至6.4%,在解耦场景下由51.7%降至2.6%。系统可扩展至超1000张GPU,基于通用CPU运行,支持新型应用场景,如依赖SLA的帕累托前沿探索、异构解耦资源分配、智能体推理调度验证及强化学习后训练重构。代码已开源:https://github.com/NetX-lab/Frontier。

原文摘要 · Abstract (English)

Modern LLM serving is no longer homogeneous or monolithic. Production systems now combine disaggregated execution, complex parallelism, runtime optimizations, and stateful workloads such as reasoning, agents, and RL rollouts. Simulation is attractive for exploring this growing design space, yet existing simulators lack the architectural completeness and decision-grade fidelity it demands. Their monolithic-replica abstractions are ill-suited to disaggregated serving, while average-case analytical proxies can distort SLA predictions and even reverse optimization conclusions. We present Frontier, a discrete-event simulator for modern LLM inference serving. Frontier features a disaggregated abstraction. It captures the structure and dynamics of modern serving systems by modeling co-location, Prefill-Decode Disaggregation (PDD), and Attention-FFN Disaggregation (AFD) with role-specific cluster workers, incorporating key runtime optimizations (e.g., CUDA Graphs, speculative decoding) within the scheduler-batch-engine loop, and supporting stateful requests for emerging workloads. It further provides accurate and generalizable predictions of computation, communication, and memory costs across diverse serving scenarios with complex workload compositions. On 16-H800 GPU testbed, Frontier achieves an average throughput error below 4%. Compared with state-of-the-art simulators, it reduces end-to-end latency error from 44.9% to 6.4% under co-location and from 51.7% to 2.6% under disaggregation. It scales to over 1K GPUs on commodity CPUs and enables new use cases such as SLA-dependent Pareto frontier exploration, heterogeneous disaggregated allocation, agentic reasoning scheduling validation, and RL post-training reconfiguration. We release Frontier at https://github.com/NetX-lab/Frontier.

大模型推理系统仿真解耦部署性能优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。