Charon能精准模拟大模型训练推理性能，助力系统优化。
Charon: A Unified and Fine-Grained Simulator for Large-Scale LLM Training and Inference
- 构建统一模块化仿真器，细粒度预测大模型性能
- 预测误差低于5.35%，大规模训练时低至3.74%
- 实测发现更优部署配置，具实际应用价值
大规模语言模型训练与推理的高效部署面临并行策略、系统优化和硬件配置等复杂设计空间的挑战。准确快速的性能仿真对验证‘假设性’方案至关重要。为此，我们提出Charon——一个统一、模块化、细粒度的仿真器，可高精度预测大模型性能。实验表明，Charon在不同模型与配置下均保持高精度，整体预测误差持续低于5.35%，在大规模GPU集群训练中更低至3.74%。在一次实际推理部署案例中，Charon发现的配置使系统吞吐量优于工程师调优基线，充分展现其真实世界价值。
原文摘要 · Abstract (English)
Deploying large-scale LLM training and inference with optimal performance is exceptionally challenging due to a complex design space of parallelism strategies, system optimizations, and hardware configurations. Accurate and rapid performance simulation is critical for guiding optimization efforts and system studies by validating "what-if" Hooker Figure hypotheses. To address this, we introduce Charon, a unified, modular, and fine-grained simulator for accurately predicting LLM performance. Experiments show Charon achieves high accuracy across different models and configurations, with an overall prediction error consistently under 5.35%, and even under 3.74% for training with a large-scale GPU cluster. In a practical inference deployment case, Charon discovered a configuration that improved system throughput over an engineering-tuned baseline, demonstrating its significant real-world value.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。