通过动态选低功耗核心，让手机运行大模型更省电。
MNN-AECS: Energy Optimization for LLM Decoding on Mobile Devices via Adaptive Core Selection
- 根据负载动态选择低功耗CPU核心以降低能耗
- 平均节能23%，且推理速度下降在可接受范围内
- 无需root权限或系统修改，适合移动设备部署
随着对本地化大语言模型（LLM）推理需求的增长，能效成为移动设备上的关键挑战。分析表明，内存受限的LLM解码阶段是主要能耗来源，而现有工作多聚焦于加速预填充阶段，忽视了能效问题。本文提出自适应能效核心选择（AECS），并集成至MNN，构建出首个无需root权限或操作系统修改的引擎级能效解决方案——MNN-AECS。该方案通过动态选择低功耗CPU核心，在保持解码速度可接受的延迟范围内显著降低能耗。在5个Android和2个iOS设备上，针对5个不同规模的主流LLM进行评估。相比原始MNN，MNN-AECS在所有7台设备和4个数据集上平均节能23%；相较于llama.cpp、executorch、mllm和MediaPipe等其他引擎，平均节能39%至78%，速度提升12%至363%。
原文摘要 · Abstract (English)
As the demand for on-device Large Language Model (LLM) inference grows, energy efficiency has become a major concern, especially for battery-limited mobile devices. Our analysis shows that the memory-bound LLM decode phase dominates energy use, and yet most existing works focus on accelerating the prefill phase, neglecting energy concerns. We introduce Adaptive Energy-Centric Core Selection (AECS) and integrate it into MNN to create the energy-efficient version, MNN-AECS, the first engine-level system solution without requiring root access or OS modifications for energy-efficient LLM decoding. MNN-AECS is designed to reduce LLM decoding energy while keeping decode speed within an acceptable slowdown threshold by dynamically selecting low-power CPU cores. MNN-AECS is evaluated across 5 Android and 2 iOS devices on 5 popular LLMs of various sizes. Compared to original MNN, MNN-AECS cuts down energy use by 23% without slowdown averaged over all 7 devices and 4 datasets. Against other engines, including llama.cpp, executorch, mllm, and MediaPipe, MNN-AECS delivers 39% to 78% energy saving and 12% to 363% speedup on average.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。