arXiv:2511.16449cs.CVcs.AI2025-11被引 6

解决视觉-语言-动作模型推理中语义与动作信息不匹配的剪枝难题

Bridging the Semantic-Action Gap in Visual Token Pruning for Efficient VLA Inference

  • 融合语义重要性与动作相关性的双路径剪枝机制
  • 在保持操作性能前提下实现最高1.99倍加速
  • 适合需要实时响应的机器人视觉推理场景

视觉-语言-动作(VLA)模型在具身智能中展现出巨大潜力,但实时部署需处理连续视觉流,计算开销大。视觉标记剪枝是主流加速方法,但直接用于VLA推理会导致操作性能严重下降。分析发现,这是由于预填充阶段与动作解码阶段的注意力模式差异:仅基于语义显著性剪枝会误删关键动作信息。为此,我们提出VLA-Pruner,一种基于VLA推理视觉需求的即插即用剪枝方法,利用机器人操作的时间连续性,从语义预填充和时序平滑的动作相关性两方面评估标记重要性,并采用先合并后筛选策略,在计算预算内保留紧凑非冗余标记。实验表明,VLA-Pruner在多个VLA架构上优于现有方法,实现最高1.99倍加速,同时保持相当的操作质量。

原文摘要 · Abstract (English)

Vision-Language-Action (VLA) models have shown great potential for embodied AI by integrating visual perception, language understanding, and action execution. In real-time deployment, these models must process continuous visual streams, incurring substantial computational overhead. Visual token pruning -- a mainstream technique for accelerating Vision-Language Models (VLMs) by retaining salient tokens while discarding redundant ones -- offers a natural candidate solution to this challenge. However, directly applying VLM-oriented pruning methods to VLA inference can cause severe degradation in manipulation performance. Our analysis attributes this degradation to a key mismatch: VLA inference exhibits distinct attention patterns between the vision-language prefill stage and the action-decode stage, so pruning based only on context-prefill semantic salience is biased toward semantic cues and may remove action-critical visual tokens. Motivated by this observation, we propose VLA-Pruner, an effective plug-and-play token pruning method grounded in the visual requirements of VLA inference, further exploiting the temporal continuity of robot manipulation. Specifically, VLA-Pruner estimates visual-token importance from both semantic prefilling and temporally smoothed action relevance, and then applies a Combine-then-Filter strategy to retain compact, non-redundant tokens under the compute budget. Experiments show that VLA-Pruner outperforms state-of-the-art approaches across multiple VLA architectures, achieving up to 1.99x speedup with comparable manipulation quality.

视觉推理模型加速机器人学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。