优化视觉语言动作模型微调,显著提升推理速度与任务成功率。
Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success

- 采用并行解码、动作分块和连续动作表示,提升推理效率。
- 在LIBERO模拟基准上将成功率从76.5%提升至97.1%,吞吐量提高26倍。
- 适用于高频率精细控制任务,适合机器人实操与高效部署场景。
近期的视觉语言动作模型(VLAs)基于预训练视觉语言模型,利用多样机器人数据集展现强大任务执行能力、语言跟随性能与语义泛化能力。尽管取得成功,这些模型在新机器人配置下仍表现不佳,需微调才能获得良好性能,但最佳微调策略尚不明确。本文以OpenVLA为基线模型,系统研究了动作解码方式、动作表示与学习目标等关键设计选择。通过实证分析,提出优化微调(OFT)方案:融合并行解码、动作分块、连续动作表示及基于L1回归的简单学习目标,全面提升推理效率、策略性能与输入输出灵活性。我们构建了OpenVLA-OFT,该模型在LIBERO模拟基准上达到新SOTA,使OpenVLA在四个任务套件上的平均成功率从76.5%提升至97.1%,动作生成吞吐量提升26倍。真实世界测试中,该微调方案使OpenVLA在双臂ALOHA机器人上成功完成高频率精细控制任务,优于其他微调后的VLAs(π_0 和 RDT-1B)以及从头训练的强模仿学习策略(Diffusion Policy 和 ACT),平均成功率最高提升15%(绝对值)。代码与预训练模型权重已开源:https://openvla-oft.github.io/
原文摘要 · Abstract (English)
Recent vision-language-action models (VLAs) build upon pretrained vision-language models and leverage diverse robot datasets to demonstrate strong task execution, language following ability, and semantic generalization. Despite these successes, VLAs struggle with novel robot setups and require fine-tuning to achieve good performance, yet how to most effectively fine-tune them is unclear given many possible strategies. In this work, we study key VLA adaptation design choices such as different action decoding schemes, action representations, and learning objectives for fine-tuning, using OpenVLA as our representative base model. Our empirical analysis informs an Optimized Fine-Tuning (OFT) recipe that integrates parallel decoding, action chunking, a continuous action representation, and a simple L1 regression-based learning objective to altogether improve inference efficiency, policy performance, and flexibility in the model's input-output specifications. We propose OpenVLA-OFT, an instantiation of this recipe, which sets a new state of the art on the LIBERO simulation benchmark, significantly boosting OpenVLA's average success rate across four task suites from 76.5% to 97.1% while increasing action generation throughput by 26$\times$. In real-world evaluations, our fine-tuning recipe enables OpenVLA to successfully execute dexterous, high-frequency control tasks on a bimanual ALOHA robot and outperform other VLAs ($π_0$ and RDT-1B) fine-tuned using their default recipes, as well as strong imitation learning policies trained from scratch (Diffusion Policy and ACT) by up to 15% (absolute) in average success rate. We release code for OFT and pretrained model checkpoints at https://openvla-oft.github.io/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。