arXiv:2607.04434cs.ROcs.AI2026-07被引 10

打造仿真与现实融合的机器人操作评估基准,全面测试通用机械臂能力。

RoboDojo: A Unified Sim-and-Real Benchmark for Comprehensive Evaluation of Generalist Robot Manipulation Policies

论文配图:RoboDojo: A Unified Sim-and-Real Benchmark for Comprehensive Evaluation of Generalist Robot Manipulation Policies
图 1 · 摘自论文原文
  • 构建42个仿真+18个真实任务,覆盖泛化、长程执行等五维能力
  • 支持云端可复现的真实世界评测,降低实验成本与门槛
  • 兼容30个策略模型,提供公开排行榜推动领域发展

通用机器人操作策略快速发展,但现有评估基准仍存在系统性局限。多数任务简单、时长短或能力覆盖不全,且仅在仿真或真实环境进行。仿真虽可大规模反馈,但忽略物理部署挑战;真实评估则成本高、耗时长且难复现。本文提出RoboDojo,一个统一的仿真-现实双环境评估基准,包含42个仿真任务和18个真实任务,涵盖多样化互补的操作能力。仿真部分评估泛化性、记忆、精度、长程执行及开放词汇指令遵循五维度,真实部分则暴露策略于复杂物理环境。通过Isaac Sim实现异构并行仿真,提供可复现的真实世界评测系统RoboDojo-RealEval,具备远程云访问、标准硬件、场景重置、评估协议与部署接口。结合XPolicyLab,策略只需一次集成即可跨仿真与真实环境评估,适配成本极低。已将30个策略接入并评估,建立公开排行榜,开展系统性性能分析。官网见http://robodojo-benchmark.com/。

原文摘要 · Abstract (English)

Generalist robot manipulation policies have advanced rapidly, yet existing benchmarks remain limited in systematically evaluating their capabilities. Many rely on simple, short-horizon, or skill-narrow tasks with limited capability coverage, and are often conducted only in simulation or only in the real world. Simulation enables scalable feedback but misses physical deployment challenges, while real-world evaluation is costly, time-consuming, and difficult to reproduce. We introduce RoboDojo, a unified sim-and-real benchmark for comprehensive evaluation of generalist robot manipulation policies. RoboDojo includes 42 simulation tasks and 18 real-world tasks covering diverse and complementary manipulation capabilities. The simulation benchmark evaluates five dimensions: generalization, memory, precision, long-horizon execution, and open-vocabulary instruction following, while the real-world benchmark exposes policies to challenging physical-world deployment conditions. RoboDojo supports scalable evaluation through heterogeneous parallel simulation in Isaac Sim and provides RoboDojo-RealEval, a reproducible real-world evaluation system with remote cloud access, standardized hardware, scene reset, evaluation protocol, and deployment interface. Together with XPolicyLab, policies can be integrated once and evaluated across simulation and real-world settings with minimal adaptation. We integrate 30 policies into XPolicyLab and evaluate them on RoboDojo, establishing a public leaderboard and systematic analysis of current policy performance. The website is available at http://robodojo-benchmark.com/.

机器人操作仿真评估真实世界基准测试

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。