arXiv:2502.02175cs.ROcs.CV2025-02NeurIPS被引 72

通过自适应缓存视觉令牌,让机器人操控模型更快更省算力。

VLA-Cache: Efficient Vision-Language-Action Manipulation via Adaptive Token Caching

  • 识别相邻帧中变化小的视觉区域,复用其计算结果
  • 在仿真和真实机器人上实现1.7倍加速,控制频率提升15%
  • 无需训练,适合需要实时响应的机器人系统部署

视觉-语言-动作(VLA)模型具备强大的多模态推理能力,可直接从视觉感知和语言指令生成动作,实现端到端控制。然而其高计算开销限制了实时机器人控制中的应用。本文提出VLA-Cache,一种无需训练的推理加速方法,通过自适应缓存和复用跨帧的静态视觉令牌来降低计算开销。利用机器人操作中的时间连续性,VLA-Cache识别相邻帧间变化最小的视觉令牌,复用其缓存的键值对表示,避免冗余计算。为保持动作精度,该方法选择性重新计算对环境敏感的任务相关令牌,确保关键视觉信息的准确性。为进一步提升效率,引入分层自适应令牌重用策略,根据解码器各层注意力集中度动态调整重用比例,优先保证关键令牌的重新计算。在两个仿真平台(LIBERO和SIMPLER)及一个真实机器人系统上的实验表明,VLA-Cache在CUDA延迟上最高实现1.7倍加速,控制频率提升15%,任务成功率损失可忽略不计。

原文摘要 · Abstract (English)

Vision-Language-Action (VLA) models have demonstrated strong multi-modal reasoning capabilities, enabling direct action generation from visual perception and language instructions in an end-to-end manner. However, their substantial computational cost poses a challenge for real-time robotic control, where rapid decision-making is essential. This paper introduces VLA-Cache, a training-free inference acceleration method that reduces computational overhead by adaptively caching and reusing static visual tokens across frames. Exploiting the temporal continuity in robotic manipulation, VLA-Cache identifies minimally changed tokens between adjacent frames and reuses their cached key-value representations, thereby circumventing redundant computations. Additionally, to maintain action precision, VLA-Cache selectively re-computes task-relevant tokens that are environmentally sensitive, ensuring the fidelity of critical visual information. To further optimize efficiency, we introduce a layer adaptive token reusing strategy that dynamically adjusts the reuse ratio based on attention concentration across decoder layers, prioritizing critical tokens for recomputation. Extensive experiments on two simulation platforms (LIBERO and SIMPLER) and a real-world robotic system demonstrate that VLA-Cache achieves up to 1.7x speedup in CUDA latency and a 15% increase in control frequency, with negligible loss on task success rate. The code and videos can be found at our project page: https://vla-cache.github.io.

机器人控制视觉语言模型推理加速缓存优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。