系统设计选择可让大模型能耗相差百倍，关键在量化、批处理与请求调度。
Understanding Efficiency: Quantization, Batching, and Serving Strategies in LLM Energy Use
- 通过量化、批处理和请求调度优化推理系统能显著降低能耗。
- 相同模型下，合理配置可使单次请求能耗降低100倍。
- 适合关注绿色AI部署与服务架构优化的研究者与工程师。
大型语言模型（LLMs）在生产环境中的部署日益增多，导致计算资源和能源消耗的重心从训练转向推理。尽管已有研究关注每次提示或每令牌的推理能耗，但本文强调系统级设计选择——如数值精度、批处理策略和请求调度——可使相同模型的能耗产生数量级差异。我们在NVIDIA H100 GPU上对LLM推理的能耗与延迟进行了详尽的实证研究，分析了量化、批量大小及服务配置（如Hugging Face的Text Generation Inference服务器）的影响。结果表明：低精度格式仅在计算密集型阶段节省能耗；批处理能提升能效，尤其在解码等内存密集阶段；结构化请求时间（到达塑造）可将单次请求能耗降低最多100倍。我们认为，可持续的LLM部署不仅依赖模型内部设计，更取决于服务栈的协同调度。研究呼吁开展阶段感知的能耗分析与系统级优化，以推动更绿色的AI服务。
原文摘要 · Abstract (English)
Large Language Models (LLMs) are increasingly deployed in production, contributing towards shifting the burden in terms of computational resources and energy demands from training to inference. While prior work has examined the energy cost of inference per prompt or per token, we highlight how \emph{system-level design choices} - such as numerical precision, batching strategy, and request scheduling - can lead to orders-of-magnitude differences in energy consumption for the same model. We perform a detailed empirical study of LLM inference energy and latency on NVIDIA H100 GPUs, analyzing the impact of quantization, batch size, and serving configuration (e.g., with Hugging Face's Text Generation Inference server). Our results reveal that lower-precision formats only yield energy gains in compute-bound regimes; that batching improves energy efficiency, especially in memory-bound phases like decoding; and that structured request timing (arrival shaping) can reduce per-request energy by up to 100 times. We argue that sustainable LLM deployment depends not only on model internals, but also on the orchestration of the serving stack. Our findings motivate phase-aware energy profiling and system-level optimizations for greener AI services.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。