减少动作令牌生成，提升机器人操控效率与部署速度。
VOTE: Vision-Language-Action Optimization with Trajectory Ensemble Voting
- 通过集成投票策略优化动作预测，减少令牌数量并提高并行度。
- 推理速度比OpenVLA快39倍，边缘设备上达46赫兹吞吐率。
- 适合需要低延迟、高效率部署的机器人控制场景。
近期大规模视觉-语言-动作（VLA）模型在自然语言引导的机器人操作任务中表现优异。然而，现有VLA模型存在两大缺陷：(i) 生成大量动作令牌，导致推理延迟高、训练成本上升；(ii) 生成动作利用率不足，可能造成性能损失。为此，我们提出一种微调框架，使VLA模型生成显著更少的动作令牌，同时具备高并行性，有效降低推理延迟与训练成本。此外，引入基于投票的集成推理优化技术，融合当前与历史动作预测，提升动作利用效率与整体性能。实验表明，本方法在性能上超越现有最优VLA模型，在边缘平台实现46赫兹吞吐率，推理速度比OpenVLA快39倍，展现出良好的可部署性。代码已开源：https://github.com/LukeLIN-web/VOTE。
原文摘要 · Abstract (English)
Recent large-scale Vision Language Action (VLA) models have shown superior performance in robotic manipulation tasks guided by natural language. However, current VLA models suffer from two drawbacks: (i) generation of massive tokens leading to high inference latency and increased training cost, and (ii) insufficient utilization of generated actions resulting in potential performance loss. To address these issues, we develop a training framework to finetune VLA models for generating significantly fewer action tokens with high parallelism, effectively reducing inference latency and training cost. Furthermore, we introduce an inference optimization technique with a novel voting-based ensemble strategy to combine current and previous action predictions, improving the utilization of generated actions and overall performance. Our results demonstrate that we achieve superior performance compared with state-of-the-art VLA models, achieving significantly higher success rates and 39$\times$ faster inference than OpenVLA with 46 Hz throughput on edge platforms, demonstrating practical deployability. The code is available at https://github.com/LukeLIN-web/VOTE.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。