用智能体驱动模拟器自动生成,加速大模型服务系统研发。
Simthesizer: An Agent-Driven Simulation Framework for LLM Serving Systems
- 构建可组合的动态图框架,统一表达完整服务流程与控制逻辑。
- 相比旧模拟器,吞吐量误差低至2.51%,性能提升最高达285倍。
- 适合快速迭代大模型服务系统的设计验证与实验部署。
系统级仿真对于探索大模型服务系统不断扩展的设计空间至关重要,而真实部署成本高昂且常不可行。然而,现代大模型服务发展速度已超过人工驱动的仿真器开发能力,新兴工作负载与机制(如智能体工作流、解耦服务)不再适配现有仿真器所依赖的单体式流水线。每个新机制都需侵入式重构,导致仿真器与实际系统间差距持续扩大。为此,我们提出Simthesizer——一个由智能体驱动的仿真框架。Simthesizer引入可组合的仿真基础设施,统一表达完整服务流程及协调其运行的控制决策,并将其构建成统一动态图。合成智能体(Synthesizer agent)在仿真特定约束与保真度验证下,将自然语言功能需求映射到该抽象层,持续演化单一共享仿真器,而非为每项功能新建仿真。在相同编码智能体和约束下,基于Simthesizer的扩展与基于vLLM的真实系统相比,平均吞吐量误差仅为2.51%,而传统仿真器为6.03%。在相同工作负载下,Simthesizer的仿真速度分别达到最先进的LLMServingSim2.0和Vidur的284.96倍和23.19倍。
原文摘要 · Abstract (English)
System-level simulation is an essential tool for exploring the rapidly expanding design space of LLM serving systems, where real deployments remain costly and often infeasible. However, modern LLM serving now evolves faster than human-driven simulator development can track, and emerging workloads and mechanisms, from agentic workflows to disaggregated serving, no longer fit the monolithic simulation pipeline that existing simulators assume. Each new mechanism therefore demands an invasive rewrite, leaving a widening development gap between deployed serving systems and the simulators that model them. To close this gap, we present Simthesizer, a framework that realizes agent-driven simulator development. Simthesizer introduces a composable simulator infrastructure that uniformly expresses the complete serving workflow, including the control decisions that coordinate it, and realizes it as a unified dynamic graph in Simthesizer simulator. Synthesizer agent, a harnessed coding agent, then lowers natural-language feature requests onto this abstraction under simulator-specific guardrails and fidelity validation, evolving one shared simulator instead of building a new one for every feature. Under the same coding agent and harnesses, extensions built on Simthesizer follow a vLLM-based real system with 2.51% average throughput error, versus 6.03% for extensions built on existing simulators. On identical workloads, Simthesizer also simulates up to 284.96x and 23.19x faster than two state-of-the-art simulators, LLMServingSim2.0 and Vidur, respectively.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。