无需实测即可估算大模型推理能耗，助力绿色AI设计。
From Tokens to Watt-hours: Analytical Energy Estimation for LLM Inference on Modern GPUs
- 基于参数量和计算流量，建立分阶段能耗模型。
- 可分解出输入、输出及各组件的能耗，支持多维度分析。
- 适合做模型对比、低碳编程和早期系统设计参考。
大型语言模型（LLM）推理的运行能耗正成为部署型AI系统环境影响的重要部分。然而，直接测量推理能耗通常需要硬件遥测、功耗仪表或特定基础设施监控，限制了其在比较研究、早期系统设计和可持续性报告中的应用。本报告提出一种基于分析建模、经实验校准的GPU级方法，可在不依赖实时测量的情况下，对NVIDIA H100类加速器上的LLM推理能耗进行估算。该估计算法结合参数量相关的Transformer FLOP统计、校准后的内存访问因子，以及针对FP16/BF16张量核心计算和高带宽内存传输的硬件特异性能耗系数。方法明确区分提示预填充与自回归解码阶段，可分别估算输入词元、输出词元及完整请求的能耗。进一步将总能耗分解为计算、参数访问、键值缓存写入和注意力读取四部分，支持分析模型规模、上下文长度和生成词元数对能耗的影响。所得估算结果并非替代物理功耗测量，而是提供透明、可复现且假设清晰的近似值，适用于模型对比、绿色编码分析及推理工作负载的设计期评估。
原文摘要 · Abstract (English)
The operational energy consumption of large language model (LLM) inference is becoming an increasingly important component of the environmental footprint of deployed AI systems. However, direct measurement of inference energy often requires hardware telemetry, power instrumentation, or infrastructure-specific monitoring, limiting its applicability in comparative studies, early-stage system design, and sustainability reporting. This report presents an analytically structured, empirically calibrated, GPU-level methodology for estimating LLM inference energy on NVIDIA H100-class accelerators without direct runtime measurement. The proposed estimator combines parameter-scaled transformer FLOP accounting, calibrated memory-traffic factors, and hardware-specific energy coefficients for FP16/BF16 tensor-core computation and high-bandwidth-memory movement. It explicitly separates prompt prefill from autoregressive decoding, enabling energy estimates for input tokens, output tokens, and complete inference requests. The methodology further decomposes total energy into compute, parameter-access, key-value-cache write, and attention-read components, allowing the scaling behavior with model size, context length, and generated-token count to be analyzed. The resulting estimates are not intended to replace physical power measurements; rather, they provide transparent, reproducible, and assumption-explicit approximations suitable for model comparison, green-coding analysis, and design-time evaluation of LLM inference workloads.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。