arXiv:2506.18088cs.ROcs.AI2025-06被引 425

用大规模仿真生成真实感数据,提升双臂机器人操作的鲁棒性。

RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulation

论文配图:RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulation
图 1 · 摘自论文原文
  • 基于多模态模型自动合成任务代码,实现可扩展的仿真数据生成。
  • 在50个任务上提升代码生成成功率10.9%,零样本模型性能提升228%。
  • 适合研究机器人仿真训练、跨域泛化与低成本实机部署的团队。

基于仿真的数据合成已成为推动现实世界机器人操作的重要范式。然而现有数据集在鲁棒双臂操作方面仍显不足,主要受限于(1)任务生成方法缺乏可扩展性,(2)仿真环境过于简化。我们提出RoboTwin 2.0,一个可扩展的自动化大规模生成多样化且真实感数据的框架,同时提供统一的双臂操作评估协议。其核心是RoboTwin-OD,包含731个实例、覆盖147类物体的对象库,附带语义和操作相关标注。在此基础上,设计了基于多模态语言模型(MLLMs)与仿真内迭代优化的专家数据合成流程,可自动生成任务级执行代码。为增强仿真到现实的迁移能力,RoboTwin 2.0在五个维度(杂乱度、光照、背景、桌面高度、语言)实施结构化领域随机化,显著提升数据多样性和策略鲁棒性。该框架应用于50个双臂任务及五种机器人形态。实验表明,代码生成成功率提升10.9%;基于合成数据加仅10个真实示范训练的VLA模型,相较10样本基线获得367%的相对提升;仅使用合成数据训练的零样本模型亦取得228%的性能增长。结果验证了RoboTwin 2.0在强化仿真到现实迁移与环境变化鲁棒性方面的有效性。我们开源数据生成器、基准测试、数据集与代码,以支持可扩展的鲁棒双臂操作研究。项目页面:https://robotwin-platform.github.io/,代码:https://github.com/robotwin-Platform/robotwin/

原文摘要 · Abstract (English)

Simulation-based data synthesis has emerged as a powerful paradigm for advancing real-world robotic manipulation. Yet existing datasets remain insufficient for robust bimanual manipulation due to (1) the lack of scalable task generation methods and (2) oversimplified simulation environments. We present RoboTwin 2.0, a scalable framework for automated, large-scale generation of diverse and realistic data, together with unified evaluation protocols for dual-arm manipulation. At its core is RoboTwin-OD, an object library of 731 instances across 147 categories with semantic and manipulation-relevant annotations. Building on this, we design an expert data synthesis pipeline that leverages multimodal language models (MLLMs) and simulation-in-the-loop refinement to automatically generate task-level execution code. To improve sim-to-real transfer, RoboTwin 2.0 applies structured domain randomization along five axes: clutter, lighting, background, tabletop height, and language, enhancing data diversity and policy robustness. The framework is instantiated across 50 dual-arm tasks and five robot embodiments. Empirically, it yields a 10.9% gain in code generation success rate. For downstream policy learning, a VLA model trained with synthetic data plus only 10 real demonstrations achieves a 367% relative improvement over the 10-demo baseline, while zero-shot models trained solely on synthetic data obtain a 228% gain. These results highlight the effectiveness of RoboTwin 2.0 in strengthening sim-to-real transfer and robustness to environmental variations. We release the data generator, benchmark, dataset, and code to support scalable research in robust bimanual manipulation. Project Page: https://robotwin-platform.github.io/, Code: https://github.com/robotwin-Platform/robotwin/.

机器人操作仿真生成双臂协作域随机化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。