边端视觉语言模型的能耗主要来自输出文本,而非视觉输入。
Seeing is Free, Speaking is Not: Uncovering the True Energy Bottleneck in Edge VLM Inference

- 通过系统实测发现,模型能耗主要由生成文本长度决定,而非图像复杂度。
- 每生成一个输出词耗时是输入词的11到39倍,显著增加延迟和能耗。
- 控制输出长度可节省高达97%能量,适合资源受限的边缘部署场景。
视觉语言模型(VLMs)是具身智能的感知核心,但其在边缘设备上的能耗仍不清晰。现有优化多聚焦减少视觉令牌,隐含假设视觉处理是主要能耗来源。我们首次系统性地对五种模型、三种架构、四种分辨率及两种硬件平台(NVIDIA RTX 3070与Jetson Orin NX)进行能耗分析。结果表明:第一,平均推理功耗为模型固有常数,受输入分辨率、图像复杂度和提示类型影响小于5%,所有能耗变化均源于推理时间差异;第二,每个输出令牌耗时为输入令牌的11至39倍,因预填充与解码阶段存在计算与内存瓶颈的不对称,输出长度成为延迟与能耗的主要驱动因素;第三,图像中物体数量差异可导致4.1倍能耗波动,但非因视觉处理成本上升,而是输出长度变化所致。这揭示了视觉令牌剪枝的局限性——即使完全移除视觉令牌,最多仅节省10%总能耗。对于参数量从10亿到80亿的模型,控制输出长度可节省高达97%能耗,且大模型下解码能耗占比更强。结论:边端VLM推理的真实能耗瓶颈不在于‘看’,而在于‘说’。
原文摘要 · Abstract (English)
Vision-Language Models (VLMs) are the perceptual backbone of embodied AI, but their energy footprint on edge hardware remains poorly understood. Existing efficiency efforts focus predominantly on reducing visual tokens, implicitly treating visual processing as the dominant energy cost. We overturn this implicit assumption through the first systematic energy profiling of on-device VLM inference, spanning five models across three architecture families, four input resolutions, and two hardware platforms (NVIDIA RTX 3070 and Jetson Orin NX). Our analysis yields three findings. First, average inference power is a model-intrinsic constant, invariant to input resolution, image complexity, and prompt type, with less than 5% variation across all conditions. This means that all energy variation across inputs must arise from variation in inference time, not from variation in power draw. Second, each output token costs 11 to 39x more wall-clock time than each input token due to the compute-bound and memory-bound asymmetry between prefill and decode, making output token count the dominant driver of both latency and energy. Third, image complexity, measured by the number of objects in an image, induces up to 4.1x energy differences at identical resolution. This variation arises not from increased visual processing cost, but from differences in output length. These findings expose a fundamental limitation of visual token pruning: even removing all visual tokens saves at most 10% of total energy for fixed-token models. Across models spanning 1 billion to 8 billion parameters, controlling output length saves up to 97% of total energy, with the energy dominance of decoding growing stronger at larger model scale. In short, the true energy bottleneck in edge VLM inference is not what the model sees, but how much it says. Code is available at https://github.com/Junfei-Z/seeing-is-free.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。