用自生成数据迭代优化视觉语言动作模型与世界模型
VLAW: Iterative Co-Improvement of Vision-Language-Action Policy and World Model
- 通过真实数据改进世界模型，再用其生成合成数据
- 在真实机器人上实现39.2%的成功率提升
- 适合做具身智能、机器人操作的研究者
本文旨在通过在线交互迭代提升视觉-语言-动作（VLA）模型的性能与可靠性。由于真实世界中收集策略轨迹成本高昂，我们探索是否可利用学习到的模拟器——特别是动作条件视频生成模型——来生成额外的轨迹数据。然而，现有世界模型缺乏足够的物理保真度：它们主要在示范数据集上训练，覆盖多种物理交互不足（尤其是失败案例），且难以准确建模接触丰富的物体操作中的细微物理细节。为此，我们提出一种简单迭代优化算法，使用真实世界轨迹数据提升世界模型保真度，进而生成补充的合成数据以优化VLA模型。在真实机器人上的实验表明，该方法显著提升了先进VLA模型在多个下游任务的表现，相比基础策略成功率提升39.2%，仅用生成合成轨迹训练即实现11.6%的提升。视频展示见匿名网站：https://sites.google.com/view/vla-w
原文摘要 · Abstract (English)
The goal of this paper is to improve the performance and reliability of vision-language-action (VLA) models through iterative online interaction. Since collecting policy rollouts in the real world is expensive, we investigate whether a learned simulator-specifically, an action-conditioned video generation model-can be used to generate additional rollout data. Unfortunately, existing world models lack the physical fidelity necessary for policy improvement: they are predominantly trained on demonstration datasets that lack coverage of many different physical interactions (particularly failure cases) and struggle to accurately model small yet critical physical details in contact-rich object manipulation. We propose a simple iterative improvement algorithm that uses real-world roll-out data to improve the fidelity of the world model, which can then, in turn, be used to generate supplemental synthetic data for improving the VLA model. In our experiments on a real robot, we use this approach to improve the performance of a state-of-the-art VLA model on multiple downstream tasks. We achieve a 39.2% absolute success rate improvement over the base policy and 11.6% improvement from training with the generated synthetic rollouts. Videos can be found at this anonymous website: https://sites.google.com/view/vla-w
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。