揭示本地大模型在CPU上的能效缩放规律,提出高效压缩与预处理方案。
Scaling Laws for Energy Efficiency of Local LLMs
- 基于真实设备测试,发现语言模型计算量随输入长度线性增长。
- 视觉语言模型存在分辨率拐点,高于阈值时计算量基本不变。
- 量子启发压缩可降低71.9%资源占用,适合边缘设备部署。
将本地大语言模型和视觉语言模型部署于边缘设备时,需在精度与受限的计算和能耗预算间取得平衡。尽管图形处理器主导现代人工智能部署,但大多数消费级硬件——包括笔记本、台式机、工业控制器和嵌入式系统——仍依赖中央处理器。然而,仅使用中央处理器进行本地语言与视觉语言任务推理的计算规律尚未被充分探索。本文系统性地在两类典型中央处理器平台(搭载M2芯片的MacBook Pro代表主流笔记本部署,搭载Raspberry Pi 5代表低功耗嵌入式场景)上基准测试大语言模型与视觉语言模型。采用统一方法,通过连续采样处理器与内存使用并结合曲线下面积积分,量化语言模型输入文本长度与计算负载的关系,以及视觉语言模型图像分辨率与计算负载的关系。研究发现两个经验缩放规律:(1) 语言模型推理的计算成本随标记长度近似线性增长;(2) 视觉语言模型呈现由预处理驱动的“分辨率拐点”,当图像分辨率超过内部阈值后计算量保持稳定,低于该阈值时则急剧下降。此外,我们证明量子启发压缩技术可使处理器与内存使用减少高达71.9%,能量消耗降低最多达62%,同时保持或提升语义准确性。这些结果为本地多模态中央处理器推理提供了系统性量化分析,并指出模型压缩与输入分辨率预处理是实现可持续边缘推理的有效、低成本手段。
原文摘要 · Abstract (English)
Deploying local large language models and vision-language models on edge devices requires balancing accuracy with constrained computational and energy budgets. Although graphics processors dominate modern artificial-intelligence deployment, most consumer hardware--including laptops, desktops, industrial controllers, and embedded systems--relies on central processing units. Despite this, the computational laws governing central-processing-unit-only inference for local language and vision-language workloads remain largely unexplored. We systematically benchmark large language and vision-language models on two representative central-processing-unit tiers widely used for local inference: a MacBook Pro M2, reflecting mainstream laptop-class deployment, and a Raspberry Pi 5, representing constrained, low-power embedded settings. Using a unified methodology based on continuous sampling of processor and memory usage together with area-under-curve integration, we characterize how computational load scales with input text length for language models and with image resolution for vision-language models. We uncover two empirical scaling laws: (1) computational cost for language-model inference scales approximately linearly with token length; and (2) vision-language models exhibit a preprocessing-driven "resolution knee", where compute remains constant above an internal resolution clamp and decreases sharply below it. Beyond these laws, we show that quantum-inspired compression reduces processor and memory usage by up to 71.9% and energy consumption by up to 62%, while preserving or improving semantic accuracy. These results provide a systematic quantification of multimodal central-processing-unit-only scaling for local language and vision-language workloads, and they identify model compression and input-resolution preprocessing as effective, low-cost levers for sustainable edge inference.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。