用混合模型精准预测大模型推理延迟与能耗，助力高效设计。
Multi-Level Modeling of Large Language Model Inference Latency and Energy via Hybrid Analytical--Machine-Learning Predictors
- 结合解析建模与机器学习，分三层预测计算开销。
- 在H100上对LLaMA 3 8B误差低于5%，覆盖预填充和解码阶段。
- 无需真实硬件即可快速探索模型设计，适合系统优化与能效研究者。
大型语言模型（LLMs）的快速扩展带来了巨大的计算成本、能耗和推理延迟，准确估算对于可持续人工智能部署和硬件感知设计至关重要。本文提出HYMELL，一种融合解析建模与机器学习的三层次框架，用于估计LLM推理延迟与能耗。该框架分层建模：第一层解析估算基础操作开销，第二层用机器学习预测高层组件性能，第三层构建端到端模型捕捉预填充与解码阶段的系统级开销。支持多种架构，包括密集型与专家混合（MoE）前馈网络（FFN），以及多头注意力（MHA）和分组查询注意力（GQA）。在NVIDIA H100 GPU上评估显示，对LLaMA 3 8B模型，预填充与解码阶段误差均低于5%。通过直接从架构参数预测执行成本，实现无需硬件的快速设计空间探索与能效优化。
原文摘要 · Abstract (English)
The rapid scaling of Large Language Models (LLMs) has significantly increased computational cost, energy consumption, and inference latency, making accurate estimation essential for sustainable artificial intelligence deployment and hardware-aware design. In this work, we introduce Hybrid Modeling for Energy and Latency of LLMs (HYMELL), a hybrid three-level framework for estimating LLM inference latency and energy by combining analytical modeling with machine learning (ML). HYMELL models LLM execution through a three-level hierarchy: analytical estimation of primitive operations, ML prediction of higher-level components, and an end-to-end model that captures system-level overheads across both prefill and decode phases. The framework supports diverse architectures, including dense and mixture-of-experts (MoE) feed-forward networks (FFNs), as well as multi-head attention (MHA) and grouped-query attention (GQA) mechanisms. Evaluated on an NVIDIA H100 graphics processing unit (GPU), HYMELL achieves high predictive accuracy; notably, for LLaMA 3 8B, it attains less than 5% error for both prefill and decode phases. By predicting execution costs directly from architectural parameters, it enables fast, hardware-free design space exploration and energy-efficient optimization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。