arXiv:2506.12723cs.CVcs.AI2025-06被引 53

通过动态调度与令牌剪枝,让视觉语言动作模型更快更准地执行。

SP-VLA: A Joint Model Scheduling and Token Pruning Approach for VLA Model Acceleration

  • 按动作类型动态切换大模型和轻量生成器,减少冗余计算。
  • 在LIBERO上提速1.5倍,SimplerEnv上提速2.4倍,精度几乎无损。
  • 适合机器人操控等实时任务,尤其关注推理效率的开发者。

视觉-语言-动作(VLA)模型因其强大的控制能力受到广泛关注,但其高计算成本和低执行频率限制了其在机器人操作、自主导航等实时任务中的应用。现有加速方法多聚焦于结构优化,忽略了这些模型在序列决策环境中的特性,导致动作生成的时序冗余与视觉输入的空间冗余未被解决。为此,本文提出SP-VLA,一种联合模型调度与令牌剪枝的统一加速框架。设计了基于动作感知的模型调度机制,通过动态切换VLA模型与轻量生成器,减少时序冗余;借鉴人类在关键决策点专注、其余依赖直觉的行为模式,将动作分为深思型与直觉型,分别交由大模型与轻量生成器处理,实现频率自适应执行。为应对空间冗余,进一步提出时空语义双感知的令牌剪枝方法,根据空间与语义重要性对令牌进行分类剪枝,加速推理。两项机制协同工作,使模型聚焦关键动作与显著视觉信息,在保持高精度的同时实现高效加速。大量实验表明,本方法在LIBERO上实现1.5×无损加速,在SimplerEnv上达2.4×,平均性能提升最高6%;推理频率与延迟在SimplerEnv中提升2.2×,在LIBERO中提升1.4×。

原文摘要 · Abstract (English)

Vision-Language-Action (VLA) models have attracted increasing attention for their strong control capabilities. However, their high computational cost and low execution frequency hinder their suitability for real-time tasks such as robotic manipulation and autonomous navigation. Existing VLA acceleration methods primarily focus on structural optimization, overlooking the fact that these models operate in sequential decision-making environments. As a result, temporal redundancy in sequential action generation and spatial redundancy in visual input remain unaddressed. To this end, we propose SP-VLA, a unified framework that accelerates VLA models by jointly scheduling models and pruning tokens. Specifically, we design an action-aware model scheduling mechanism that reduces temporal redundancy by dynamically switching between VLA model and a lightweight generator. Inspired by the human motion pattern of focusing on key decision points while relying on intuition for other actions, we categorize VLA actions into deliberative and intuitive, assigning the former to the VLA model and the latter to the lightweight generator, enabling frequency-adaptive execution through collaborative model scheduling. To address spatial redundancy, we further develop a spatio-semantic dual-aware token pruning method. Tokens are classified into spatial and semantic types and pruned based on their dual-aware importance to accelerate VLA inference. These two mechanisms work jointly to guide the VLA in focusing on critical actions and salient visual information, achieving effective acceleration while maintaining high accuracy. Extensive experiments show that our method achieves 1.5$\times$ lossless acceleration in LIBERO and 2.4$\times$ in SimplerEnv, with up to 6% average performance gain. Inference frequency and latency improve by 2.2$\times$ in SimplerEnv and 1.4$\times$ in LIBERO.

模型加速机器人控制视觉语言动作令牌剪枝

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。