arXiv:2602.23036cs.DCcs.AI2026-02被引 10

打造统一仿真器,解析大模型服务中软硬件协同瓶颈。

LLMServingSim 2.0: A Unified Simulator for Heterogeneous and Disaggregated LLM Serving Infrastructure

  • 构建运行时驱动的统一仿真框架,显式建模软硬件交互。
  • 实测误差仅0.95%,复杂配置下仿真耗时约10分钟。
  • 支持新加速器与内存系统扩展,适合架构师和系统开发者。

大语言模型(LLM)服务基础设施正向异构化与解耦化演进。现代部署越来越多地融合多样加速器与近存处理技术,引入显著的硬件异构性;同时系统软件日益将计算、内存与模型组件分离至分布式资源,以提升可扩展性与效率。因此,LLM服务性能不再由硬件或软件单独决定,而是取决于调度、数据移动与互连行为带来的运行时交互。然而,现有仿真器缺乏在统一、运行时驱动框架中联合建模异构硬件与解耦服务技术的能力。本文提出LLMServingSim 2.0,一个面向异构与解耦LLM服务基础设施的系统级统一仿真器,旨在显式表达并分析软硬件间的运行时交互。该仿真器将服务决策与硬件行为嵌入单一运行时循环,实现对批处理、路由、卸载、内存与功耗的交互感知建模。通过基于性能剖析的建模方式,支持新兴加速器与内存系统的可扩展集成,同时捕捉动态服务行为与系统级效应。我们在真实部署上验证了该仿真器,结果表明其能以平均0.95%的误差复现关键性能、内存与功耗指标,即使在复杂配置下,仿真时间也保持在约10分钟。这些成果证明,LLMServingSim 2.0为硬件创新与服务系统设计之间搭建了实用桥梁,支持下一代LLM服务基础设施的系统性探索与协同设计。

原文摘要 · Abstract (English)

Large language model (LLM) serving infrastructures are undergoing a shift toward heterogeneity and disaggregation. Modern deployments increasingly integrate diverse accelerators and near-memory processing technologies, introducing significant hardware heterogeneity, while system software increasingly separates computation, memory, and model components across distributed resources to improve scalability and efficiency. As a result, LLM serving performance is no longer determined by hardware or software choices in isolation, but by their runtime interaction through scheduling, data movement, and interconnect behavior. However, understanding these interactions remains challenging, as existing simulators lack the ability to jointly model heterogeneous hardware and disaggregated serving techniques within a unified, runtime-driven framework. This paper presents LLMServingSim 2.0, a unified system-level simulator designed to make runtime-driven hardware-software interactions in heterogeneous and disaggregated LLM serving infrastructures explicit and analyzable. LLMServingSim 2.0 embeds serving decisions and hardware behavior into a single runtime loop, enabling interaction-aware modeling of batching, routing, offloading, memory, and power. The simulator supports extensible integration of emerging accelerators and memory systems through profile-based modeling, while capturing dynamic serving behavior and system-level effects. We validate LLMServingSim 2.0 against real deployments, showing that it reproduces key performance, memory, and power metrics with an average error of 0.95%, while maintaining simulation times of around 10 minutes even for complex configurations. These results demonstrate that LLMServingSim 2.0 provides a practical bridge between hardware innovation and serving-system design, enabling systematic exploration and co-design for next-generation LLM serving infrastructures.

大模型服务系统仿真异构计算

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。