arXiv:2512.20276cs.AIcs.RO2025-12被引 5

让视觉语言模型在边缘设备上实现实时动态操作。

ActionFlow: A Pipelined Action Acceleration for Vision Language Models on Edge

  • 通过跨请求流水线调度,合并计算与内存操作提升硬件利用率。
  • 在不重新训练的情况下,将推理速度提升2.55倍至15.6 FPS。
  • 适合需要低延迟响应的机器人控制场景,如边缘端实时操作。

视觉-语言-动作(VLA)模型已成为机器人感知与控制的统一范式,支持涌现式泛化和长时序任务执行。然而,其在动态真实环境中的部署受限于高推理延迟:当前VLA模型在边缘设备上通常仅能达到3-5 FPS,远低于机器人交互所需的20-30 Hz控制频率,主要因自回归解码的内存瓶颈所致。现有优化方法常需大量重训练或牺牲模型精度。为此,我们提出ActionFlow,一种面向资源受限边缘平台的系统级推理框架。核心是跨请求流水线策略,将VLA推理重构为微请求的宏观流水线,智能地将连续时间步中的内存密集型解码阶段与计算密集型预填充阶段进行批处理,最大化硬件利用率。此外,我们设计了跨请求状态打包前向运算器与统一的KV环形缓冲区,将碎片化的内存操作融合为高效稠密计算。实验表明,ActionFlow在OpenVLA-7B模型上实现2.55倍的帧率提升(从6.1提升至15.6 FPS),无需重训练,实现了边缘设备上的实时动态操控。代码已公开于https://anonymous.4open.science/r/ActionFlow-1D47。

原文摘要 · Abstract (English)

Vision-Language-Action (VLA) models have emerged as a unified paradigm for robotic perception and control, enabling emergent generalization and long-horizon task execution. However, their deployment in dynamic, real-world environments is severely hin dered by high inference latency. While smooth robotic interaction requires control frequencies of 20 to 30 Hz, current VLA models typi cally operate at only 3-5 Hz on edge devices due to the memory bound nature of autoregressive decoding. Existing optimizations often require extensive retraining or compromise model accuracy. To bridge this gap, we introduce ActionFlow, a system-level inference framework tailored for resource-constrained edge plat forms. At the core of ActionFlow is a Cross-Request Pipelin ing strategy, a novel scheduler that redefines VLA inference as a macro-pipeline of micro-requests. The strategy intelligently batches memory-bound Decode phases with compute-bound Prefill phases across continuous time steps to maximize hardware utilization. Furthermore, to support this scheduling, we propose a Cross Request State Packed Forward operator and a Unified KV Ring Buffer, which fuse fragmented memory operations into efficient dense computations. Experimental results demonstrate that ActionFlow achieves a 2.55x improvement in FPS on the OpenVLA-7B model without retraining, enabling real-time dy namic manipulation on edge hardware. Our work is available at https://anonymous.4open.science/r/ActionFlow-1D47.

边缘计算机器人控制推理加速

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。