通过层间排名一致性动态筛选视觉令牌，显著提升视觉语言动作模型效率
Beyond Attention Magnitude: Leveraging Inter-layer Rank Consistency for Efficient Vision-Language-Action Models
- 基于层间令牌排名一致性设计动态筛选机制，不依赖固定注意力阈值
- 在CogACT+SIMPLER上减少78%令牌使用量，成功率提升6%
- 无需额外训练，适配多种解码器和任务场景
视觉-语言-动作（VLA）模型在机器人操作中表现优异，但因处理密集视觉令牌导致推理延迟严重。现有令牌压缩方法主要依赖注意力幅度进行静态选择。本文挑战这一假设，发现高注意力令牌具有任务相关性，甚至可能损害策略性能。为此，提出基于tau引导的层间高效筛选框架TIES，通过自适应平衡注意力幅度与层间排名一致性，实现无需额外训练的鲁棒令牌选择。在CogACT + SIMPLER基准测试中，TIES将平均成功率提升6%，同时减少78%的令牌使用量，并展现出对多种解码器和基准的强泛化能力。
原文摘要 · Abstract (English)
Vision-Language-Action (VLA) models excel in robotic manipulation but suffer from significant inference latency due to processing dense visual tokens. Existing token reduction methods predominantly rely on attention magnitude as a static selection. In this work, we challenge this assumption, revealing that high-attention tokens are task-dependent and can even degrade policy performance. To address this, we introduce \textbf{TIES} (\textbf{T}au-guided \textbf{I}nter-layer \textbf{E}fficient \textbf{S}election), a dynamic framework guided by inter-layer token ranking consistency. By adaptively balancing attention magnitude with ranking consistency, TIES ensures robust token selection without requiring additional training. On the CogACT + SIMPLER benchmark, TIES improves average success rates by 6\% while reducing token usage by 78\%, and demonstrate strong generalization across diverse decoders and benchmarks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。