用智能体自动生成多样化真实机器人任务,提升视觉语言模型预训练效果
RoboGene: Boosting VLA Pre-training via Diversity-Driven Agentic Framework for Real-World Task Generation
- 设计智能体框架,通过多样性采样与自我反思生成合理操作任务
- 收集1.8万条真实轨迹,显著优于GPT-4o等模型生成的任务质量
- 适合研究机器人具身智能、多模态预训练的学者与开发者
通用机器人操作的发展受限于多样且真实的交互数据稀缺。与视觉或语言领域从网络获取数据不同,机器人数据采集是需耗费高昂物理成本的主动过程。因此,自动化任务设计以最大化数据价值成为关键但未被充分探索的挑战。现有手动方法难以扩展且偏向常见任务,而通用大模型常生成不切实际的操作指令。为此,我们提出RoboGene,一种智能体框架,可自动生成适用于单臂、双臂及移动机器人的多样化、物理可行的操作任务。该框架整合三项核心组件:多样性驱动采样以覆盖广泛任务空间、自我反思机制以满足物理约束、人机协同优化实现持续改进。我们开展大规模定量分析与真实世界实验,收集了1.8万条轨迹数据,并引入新指标评估任务质量、可行性与多样性。结果表明,RoboGene显著优于当前主流大模型(如GPT-4o、Gemini 2.5 Pro)。真实实验还显示,使用RoboGene生成数据预训练的视觉语言模型(VLA)在任务成功率与泛化能力上均表现更优,凸显高质量任务生成的重要性。项目地址:https://robogene-boost-vla.github.io。
原文摘要 · Abstract (English)
The pursuit of general-purpose robotic manipulation is hindered by the scarcity of diverse, real-world interaction data. Unlike data collection from web in vision or language, robotic data collection is an active process incurring prohibitive physical costs. Consequently, automated task curation to maximize data value remains a critical yet under-explored challenge. Existing manual methods are unscalable and biased toward common tasks, while off-the-shelf foundation models often hallucinate physically infeasible instructions. To address this, we introduce RoboGene, an agentic framework designed to automate the generation of diverse, physically plausible manipulation tasks across single-arm, dual-arm, and mobile robots. RoboGene integrates three core components: diversity-driven sampling for broad task coverage, self-reflection mechanisms to enforce physical constraints, and human-in-the-loop refinement for continuous improvement. We conduct extensive quantitative analysis and large-scale real-world experiments, collecting datasets of 18k trajectories and introducing novel metrics to assess task quality, feasibility, and diversity. Results demonstrate that RoboGene significantly outperforms state-of-the-art foundation models (e.g., GPT-4o, Gemini 2.5 Pro). Furthermore, real-world experiments show that VLA models pre-trained with RoboGene achieve higher success rates and superior generalization, underscoring the importance of high-quality task generation. Our project is available at https://robogene-boost-vla.github.io.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。