用硬件模拟器分析大模型性能，发现并优化了通信瓶颈。
Fake Runs, Real Fixes -- Analyzing xPU Performance Through Simulation
- 用生产环境代码在硬件模拟器中重放，实现机器码级性能分析。
- 发现通信集体操作的微架构缺陷，优化后提升15%效率、降低4.1%延迟。
- 适合需要极致优化推理性能的工程师和系统研发团队。
随着模型规模增大，机器学习加速器成为稀缺资源，其性能需持续优化以提升效率。现有性能分析工具粒度粗，无法捕捉模型在机器码层面的表现，且常缺乏具体优化建议。本文提出xPU-Shark，一种细粒度的机器码级分析方法，可提供可执行的优化建议。核心思路是复用硬件设计过程中的硬件级模拟器，从生产部署中捕获执行轨迹，并在修改后的微架构模拟器中重放，获得模型性能的底层洞察。我们为自研加速器实现了xPU-Shark，用于分析多个上线的大语言模型，揭示了此前未知的微架构效率问题。基于这些发现，我们对常见通信集体操作进行了优化，性能提升达15%，令牌生成延迟降低最多4.1%。
原文摘要 · Abstract (English)
As models become larger, ML accelerators are a scarce resource whose performance must be continually optimized to improve efficiency. Existing performance analysis tools are coarse grained, and fail to capture model performance at the machine-code level. In addition, these tools often do not provide specific recommendations for optimizations. We present xPU-Shark, a fine-grained methodology for analyzing ML models at the machine-code level that provides actionable optimization suggestions. Our core insight is to use a hardware-level simulator, an artifact of the hardware design process that we can re-purpose for performance analysis. xPU-Shark captures traces from production deployments running on accelerators and replays them in a modified microarchitecture simulator to gain low-level insights into the model's performance. We implement xPU-Shark for our in-house accelerator and used it to analyze the performance of several of our production LLMs, revealing several previously-unknown microarchitecture inefficiencies. Leveraging these insights, we optimize a common communication collective by up to 15% and reduce token generation latency by up to 4.1%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。