arXiv:2409.12514cs.ROcs.CV2024-09被引 424

TinyVLA让机器人模型更快更省数据,无需预训练就能高效执行指令。

TinyVLA: Towards Fast, Data-Efficient Vision-Language-Action Models for Robotic Manipulation

  • 用高效多模态模型初始化策略,直接跳过预训练阶段。
  • 推理速度比OpenVLA快3倍以上,仅需少量数据即可达到相似性能。
  • 适合快速部署于真实机器人,尤其适用于资源受限场景。

视觉-语言-动作(VLA)模型在端到端学习中展现出强大的视觉运动控制与指令理解能力。然而,现有VLA模型推理缓慢且需大量机器人数据预训练,难以实际部署。本文提出紧凑型VLA模型TinyVLA,具备两大优势:(1)推理更快,(2)数据效率更高,无需预训练。其框架包含两个核心组件:(1)使用高性能、高吞吐的多模态模型初始化策略主干;(2)在微调阶段引入扩散策略解码器,实现精确机器人动作生成。我们在仿真和真实机器人上进行了广泛评估,结果表明TinyVLA在速度和数据效率方面显著优于当前最先进的OpenVLA模型,同时性能相当或更优。此外,TinyVLA在语言指令、新物体、未见位置、外观变化、背景差异及环境迁移等维度均表现出强泛化能力,常达到甚至超过OpenVLA水平。我们相信该方法为利用预训练多模态模型进行策略学习提供了新思路。

原文摘要 · Abstract (English)

Vision-Language-Action (VLA) models have shown remarkable potential in visuomotor control and instruction comprehension through end-to-end learning processes. However, current VLA models face significant challenges: they are slow during inference and require extensive pre-training on large amounts of robotic data, making real-world deployment difficult. In this paper, we introduce a new family of compact vision-language-action models, called TinyVLA, which offers two key advantages over existing VLA models: (1) faster inference speeds, and (2) improved data efficiency, eliminating the need for pre-training stage. Our framework incorporates two essential components to build TinyVLA: (1) initializing the policy backbone with robust, high-speed multimodal models, and (2) integrating a diffusion policy decoder during fine-tuning to enable precise robot actions. We conducted extensive evaluations of TinyVLA in both simulation and on real robots, demonstrating that our approach significantly outperforms the state-of-the-art VLA model, OpenVLA, in terms of speed and data efficiency, while delivering comparable or superior performance. Additionally, TinyVLA exhibits strong generalization capabilities across various dimensions, including language instructions, novel objects, unseen positions, changes in object appearance, background variations, and environmental shifts, often matching or exceeding the performance of OpenVLA. We believe that \methodname offers an interesting perspective on utilizing pre-trained multimodal models for policy learning. Our project is at https://tiny-vla.github.io.

机器人视觉语言高效模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。