根据动作动态智能剪枝视觉特征,让机器人操作更省力高效
Action-aware Dynamic Pruning for Efficient Vision-Language-Action Manipulation
- 按动作变化程度动态调整视觉信息保留比例
- 在LIBERO上实现1.35倍加速,成功率提升25.8%
- 适合需要长时间视觉推理的机器人控制场景
基于视觉-语言-动作模型的机器人操作需在长时序多模态上下文中高效推理,其中对密集视觉标记的注意力占据主要计算开销。现有方法通过减少视觉冗余提升速度,但忽略了不同操作阶段冗余度差异。我们发现粗粒度操作阶段的视觉冗余高于精细操作阶段,且与动作动态强相关。为此提出行动感知动态剪枝(ADP),融合文本驱动标记选择与动作感知轨迹门控。该方法引入门控机制,以近期动作轨迹为条件生成剪枝信号,利用历史运动窗口自适应调整各阶段的标记保留率,在计算效率与感知精度间取得平衡。在LIBERO基准和多种真实场景中的实验表明,该方法显著降低浮点运算量与动作推理延迟(如OpenVLA-OFT上提速1.35倍),同时保持较高成功率(相比基线提升25.8%),为高效机器人策略提供即插即用方案,推动了机器人操作的效率与性能边界。
原文摘要 · Abstract (English)
Robotic manipulation with Vision-Language-Action models requires efficient inference over long-horizon multi-modal context, where attention to dense visual tokens dominates computational cost. Existing methods optimize inference speed by reducing visual redundancy within VLA models, but they overlook the varying redundancy across robotic manipulation stages. We observe that the visual token redundancy is higher in coarse manipulation phase than in fine-grained operations, and is strongly correlated with the action dynamic. Motivated by this observation, we propose \textbf{A}ction-aware \textbf{D}ynamic \textbf{P}runing (\textbf{ADP}), a multi-modal pruning framework that integrates text-driven token selection with action-aware trajectory gating. Our method introduces a gating mechanism that conditions the pruning signal on recent action trajectories, using past motion windows to adaptively adjust token retention ratios in accordance with dynamics, thereby balancing computational efficiency and perceptual precision across different manipulation stages. Extensive experiments on the LIBERO suites and diverse real-world scenarios demonstrate that our method significantly reduces FLOPs and action inference latency (\textit{e.g.} $1.35 \times$ speed up on OpenVLA-OFT) while maintaining competitive success rates (\textit{e.g.} 25.8\% improvements with OpenVLA) compared to baselines, thereby providing a simple plug-in path to efficient robot policies that advances the efficiency and performance frontier of robotic manipulation. Our project website is: \href{https://vla-adp.github.io/}{ADP.com}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。