开源模块化框架,统一视觉-语言-动作模型开发与评估
StarVLA: A Lego-like Codebase for Vision-Language-Action Model Developing
- 模块化设计支持不同主干网络与动作解码器自由组合
- 多基准测试统一接口,训练效果媲美甚至超越已有方法
- 适合研究者快速复现和开发新智能体系统
构建通用具身智能体需融合感知、语言理解与行动能力,这正是基于多模态基础模型的视觉-语言-动作(VLA)方法的核心目标。尽管进展迅速,现有VLA方法在架构、代码库与评估协议上仍高度碎片化,阻碍了可比性与可复现性。本文提出StarVLA,一个开源的VLA研究代码库。它通过三项改进解决此问题:第一,采用模块化骨干-动作头架构,支持如Qwen-VL等视觉语言模型与Cosmos等世界模型骨干,并允许独立替换;第二,提供跨具身学习与多模态协同训练等可复用的训练策略;第三,集成LIBERO、SimplerEnv、RoboTwin~2.0、RoboCasa-GR1和BEHAVIOR-1K等多个主流基准,通过统一接口支持仿真与真实机器人部署。StarVLA还提供简单可复现的单基准训练配方,无需复杂数据工程即可在多个基准上达到或超越先前方法表现,无论使用哪种骨干网络。据我们所知,StarVLA是当前最全面的开源VLA框架之一,有望降低方法复现与新模型原型设计门槛。项目持续维护更新,代码与文档见https://github.com/starVLA/starVLA。
原文摘要 · Abstract (English)
Building generalist embodied agents requires integrating perception, language understanding, and action, which are core capabilities addressed by Vision-Language-Action (VLA) approaches based on multimodal foundation models, including recent advances in vision-language models and world models. Despite rapid progress, VLA methods remain fragmented across incompatible architectures, codebases, and evaluation protocols, hindering principled comparison and reproducibility. We present StarVLA, an open-source codebase for VLA research. StarVLA addresses these challenges in three aspects. First, it provides a modular backbone--action-head architecture that supports both VLM backbones (e.g., Qwen-VL) and world-model backbones (e.g., Cosmos) alongside representative action-decoding paradigms, all under a shared abstraction in which backbone and action head can each be swapped independently. Second, it provides reusable training strategies, including cross-embodiment learning and multimodal co-training, that apply consistently across supported paradigms. Third, it integrates major benchmarks, including LIBERO, SimplerEnv, RoboTwin~2.0, RoboCasa-GR1, and BEHAVIOR-1K, through a unified evaluation interface that supports both simulation and real-robot deployment. StarVLA also ships simple, fully reproducible single-benchmark training recipes that, despite minimal data engineering, already match or surpass prior methods on multiple benchmarks with both VLM and world-model backbones. To our best knowledge, StarVLA is one of the most comprehensive open-source VLA frameworks available, and we expect it to lower the barrier for reproducing existing methods and prototyping new ones. StarVLA is being actively maintained and expanded; we will update this report as the project evolves. The code and documentation are available at https://github.com/starVLA/starVLA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。