EnergyLens可精准预测多卡大模型推理能耗,助力节能部署决策。
EnergyLens: Predictive Energy-Aware Exploration for Multi-GPU LLM Inference Optimization

- 基于einsum接口建模融合、并行与通信重叠,支持多卡能耗预测。
- 多卡预填充与解码能耗预测误差仅9.25%~13.19%,跨SM分配误差12.97%。
- 揭示配置间能耗差异达1.47倍至52.9倍,适合大规模部署优化参考。
我们提出EnergyLens,一个端到端的能量感知大语言模型(LLM)推理优化框架。随着LLM规模扩大,预测和降低其能耗对可持续发展和数据中心运营至关重要,但现有方法要么需要生产级代码和昂贵的性能分析,要么无法准确捕捉多卡能耗行为。因此,从业者缺乏工具来决定优先优化方向或在无法全面分析时选择部署配置。EnergyLens通过直观的einsum接口,建模包括融合、并行及计算-通信重叠在内的LLM特性,结合负载不均衡感知的MoE建模与基于实证的多卡通信能耗模型,填补了这一空白。我们在Llama3和Qwen3-MoE上验证了该框架,针对张量并行和专家并行配置,多卡预填充与解码能耗预测的平均绝对百分比误差(MAPE)为9.25%至13.19%,梅格拉顿式重叠下的跨SM分配误差为12.97%。能源驱动的探索发现,不同配置间预填充与解码效率的能耗差异最高可达1.47倍和52.9倍,推动分布式服务设计。此外,我们证明仅凭直觉难以优化计算-通信重叠,而EnergyLens能正确识别帕累托最优配置。
原文摘要 · Abstract (English)
We present EnergyLens, an end-to-end framework for energy-aware large language model (LLM) inference optimization. As LLMs scale, predicting and reducing their energy footprint has become critical for sustainability and datacenter operations, yet existing approaches either require production-level code and expensive profiling or fail to accurately capture multi-GPU energy behavior. As a result, practitioners lack tools for deciding which optimizations to prioritize and for selecting among existing deployment configurations when exhaustive profiling is impractical. EnergyLens addresses this gap with an intuitive einsum-based interface that captures LLM specifications including fusion, parallelism, and compute-communication overlap, combined with load-imbalance-aware MoE modeling and an empirically driven communication energy model for multi-GPU settings. We validate EnergyLens on Llama3 and Qwen3-MoE across tensor-parallel and expert-parallel configurations, achieving mean absolute percentage errors (MAPEs) between 9.25% and 13.19% for multi-GPU prefill and decode energy, and 12.97% across SM allocations for Megatron-style overlap. Our energy-driven exploration reveals up to 1.47x and 52.9x energy variation across configurations in prefill and decode efficiency and motivates distributed serving. We further show that compute-communication overlap is difficult to optimize with intuition alone, but EnergyLens correctly identifies Pareto-optimal overlap configurations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。