arXiv:2606.23001cs.SEcs.LG2026-06

让手机端大模型推理更省电不发热,还能保持流畅体验。

EnerInfer: Energy-Aware On-Device LLM Inference

论文配图:EnerInfer: Energy-Aware On-Device LLM Inference
图 1 · 摘自论文原文
  • 根据模型结构预测不同频率下的性能与功耗,智能选配置。
  • 实测在手机/笔记本/开发板上节能最高达65%,且体验不下降。
  • 无需复杂传感器,适合真实设备部署,尤其适合移动场景。

设备端大模型推理因隐私保护、可靠性与成本优势日益受关注,但其能耗与温升仍是主要瓶颈。现有系统多优化解码速度,隐含假设越快越好。我们发现设备端推理存在可利用的配置冗余:适度降低NPU与内存频率,在不损害用户体验(QoE)前提下显著提升能效并减少发热。但在实际中实现这一目标极具挑战:最优能效配置随模型、推理引擎、平台和运行时条件变化,无稳定排序;商用设备缺乏组件级功耗感知,壳温也随请求到达、响应长度和热历史动态变化。为此,我们提出EnerInfer,首个联合管理能效、吞吐与热舒适的设备端大模型推理框架。它用解耦的、基于模型结构的预测与排名驱动的在线反馈,替代传统依赖模型特调与高密度传感的方案,可预测未见过的大模型在不同NPU/DDR频率设置下的吞吐与功耗,选择满足QoE的高效配置,并通过轻量级有限时域热预测,动态切换至节能或限热模式。在真实大模型上的评估显示,EnerInfer在手机、笔记本和开发板上分别实现最高65%、12%和24%的能效提升,且无QoE违规。

原文摘要 · Abstract (English)

On-device LLM inference is increasingly attractive for privacy-preserving, reliable, and cost-effective deployment, yet its energy and thermal costs remain a critical bottleneck. Existing systems primarily optimize for decoding speed, implicitly assuming that faster execution is always preferable. We show instead that on-device LLM inference often has exploitable configuration slack: modestly lowering NPU and memory frequencies preserves quality of experience (QoE) while substantially improving energy efficiency and reducing heat. Realizing this opportunity in production is challenging. The most energy-efficient NPU/DDR setting varies with the model, inference engine, platform, and runtime conditions, with no stable ranking across configurations. Commercial devices further lack component-level power sensing, and shell temperature evolves with request arrivals, response lengths, and thermal history. To address these challenges, we propose EnerInfer, the first on-device LLM inference framework that jointly manages energy efficiency, throughput, and thermal comfort for LLM workloads. EnerInfer replaces per-model profiling and sensor-heavy control with disaggregated, model-structure-aware prediction and ranking-driven online feedback. It predicts throughput and power for unseen LLMs across NPU/DDR frequency settings, selects QoE-satisfying efficient configurations under runtime interference, and uses lightweight limited-horizon thermal prediction to dynamically switch between energy-optimized and thermally constrained inference. Evaluations on real-world LLMs show that EnerInfer improves energy efficiency by up to 65%, 12%, and 24% on phones, a laptop, and a development board, respectively, without QoE violation.

大模型推理能效优化设备端部署热管理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。