通过分组路由压缩保留的多模态令牌表示,提升视觉-语言-动作模型效率。
Role-Conditioned Sub-Token Routing for Efficient Vision-Language-Action Policies
- 根据角色和上下文动态选择保留哪些子令牌分组,实现精细化压缩。
- 在4个LIBERO任务中,9.2%~11.3%的原始键值开销下仍保持强控制性能。
- 适合需要高效部署的机器人视觉-语言-动作系统研究者。
视觉-语言-动作(VLA)模型处理长序列多模态标记,推理成本高。现有方法主要减少视觉标记,但激进剪枝会丢失完整表征。子令牌压缩可保留更多标记而减小值宽。然而直接应用于VLA策略效果有限,因感知、语言理解与控制的信息分布不同。我们提出角色条件子令牌路由(RoleSub),学习如何压缩保留标记的值表示。在视觉标记减少后,RoleSub将每个保留值表示划分为正交空间中的组,并使用轻量路由器决定保留哪些组。路由决策基于标记表示、学习的隐式角色表示和语言上下文。该机制也可用于语言值压缩,实现视觉与语言表示无额外标记移除的压缩。我们在OpenVLA-OFT-7B上评估了四个LIBERO套件。在匹配视觉键值预算下,RoleSub在36项设置中优于训练过的仅令牌控制33次,尤其在激进压缩下增益最大。结合视觉与语言压缩,总键值降至原始的9.2%~11.3%,多数任务仍保持良好控制性能。结果表明,在保留标记内压缩是激进VLA压缩的有效补充。
原文摘要 · Abstract (English)
Vision-Language-Action (VLA) models process long multimodal token sequences, making inference expensive in both memory and computation. Existing efficiency methods mainly reduce visual tokens, but aggressive token pruning becomes fragile because removing a token discards its entire representation. Sub-token compression provides a complementary alternative by retaining more tokens while reducing their value width. However, directly applying sub-token compression to VLA policies is less effective because information important for perception, language understanding, and control is distributed differently across the multimodal representation. We introduce Role-Conditioned Sub-Token Routing (RoleSub), which learns how to compress the value representations of retained tokens. After visual token reduction, RoleSub partitions each retained value representation into groups in an orthogonal space and uses a lightweight router to determine which groups should be preserved. The routing decision is conditioned on the token representation, a learned latent role representation, and language context. The same mechanism can also be applied to language values, allowing visual and language representations to be compressed without removing additional tokens. We evaluate RoleSub on OpenVLA-OFT-7B across the four LIBERO suites. At matched visual-KV budgets, RoleSub outperforms a trained token-only control in 33 of 36 settings, with the largest gains under aggressive compression. Combining visual and language compression reduces total KV to 9.2--11.3% of the original while retaining strong control performance on most tasks. These results show that reducing the representation within retained tokens provides an effective complement to token pruning for aggressive VLA compression.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。