通过可微网格采样实现视觉令牌压缩，兼顾效率与精度
See What Matters: Differentiable Grid Sample Pruning for Generalizable Vision-Language-Action Model
- 设计可微网格采样器，动态选择关键视觉坐标并插值特征
- 仅用不到10%原始视觉令牌，实现76%计算量降低且成功率不变
- 适合需要高效部署的机器人视觉-语言-动作系统
视觉-语言-动作（VLA）模型在机器人操作中展现出巨大潜力，但其高计算成本阻碍了实时部署。现有令牌剪枝方法存在根本性权衡：激进压缩会丢失接触点等关键几何信息，导致性能严重下降，迫使压缩率受限，影响加速潜力。我们提出将压缩重新理解为视觉编码器中的几何感知连续令牌重采样。为此，设计了可插拔的可微网格采样器（GridS），通过任务感知的连续重采样，在保留重要空间信息的同时实现大幅压缩（少于10%原始视觉令牌）。在LIBERO基准和真实机器人平台上的实验表明，这是迄今为止最低可行视觉令牌数量的验证，实现了76%的浮点运算量（FLOPs）减少，且成功率无下降。代码已公开于https://github.com/Fediory/Grid-Sampler。
原文摘要 · Abstract (English)
Vision-Language-Action (VLA) models have shown remarkable promise in robotics manipulation, yet their high computational cost hinders real-time deployment. Existing token pruning methods suffer from a fundamental trade-off: aggressive compression using pruning inevitably discards critical geometric details like contact points, leading to severe performance degradation. This forces a compromise, limiting the achievable compression rate and thus the potential speedup. We argue that breaking this trade-off requires rethinking compression as a geometry-aware, continuous token resampling in the vision encoder. To this end, we propose the Differentiable Grid Sampler (GridS), a plug-and-play module that performs task-aware, continuous resampling of visual tokens in VLA. By adaptively predicting a minimal set of salient coordinates and extracting features via differentiable interpolation, GridS preserves essential spatial information while achieving drastic compression (with fewer than 10% original visual tokens). Experiments on both LIBERO benchmark and a real robotic platform demonstrate that validating the lowest feasible visual token count reported to date, GridS achieves a 76% reduction in FLOPs with no degradation in the success rate. The code is available at https://github.com/Fediory/Grid-Sampler.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。