arXiv:2606.16776cs.RO2026-06

用仿真打通人-机-模拟闭环,实现高效可信的机器人数据与评估

JoyAI-Sim: A Simulation-Enabled Interconversion Toolchain for the Embodied Data Pyramid

论文配图:JoyAI-Sim: A Simulation-Enabled Interconversion Toolchain for the Embodied Data Pyramid
图 1 · 摘自论文原文
  • 构建人-机-仿真双向转换链,支持真实任务数字孪生
  • 通过人类反馈优化仿真动作自然度,提升数据真实性
  • 可复用云端服务,适合机器人数据生成与模型评估团队

通用机器人策略需要可靠评估和可用训练数据,但仅靠物理机器人难以规模化。真实机器人实验虽最贴近部署场景,却耗时、昂贵且难复现。本文提出 JoyAI-Sim,一个基于仿真的双向转换工具链,实现人-机器人-仿真三者对齐。一方面,机器人→仿真→人路径将真实桌面整理任务重建为校准的数字孪生,用于可扩展评估,并借助人类具身反馈检验与优化仿真动作自然性;另一方面,人→仿真→机器人路径将人类视角演示迁移至仿真环境,经机器人物理约束验证后转化为机器人中心的轨迹、标注与视觉观测。系统以 JoySim 仿真器为核心,既作为可扩展评估层,也作为数据生成的物理一致性过滤器。核心模块(重建、仿真、渲染、真实感增强)已封装为京东云上的可复用服务,形成可扩展的机器人数据生成与模型评估基础设施。

原文摘要 · Abstract (English)

Generalist robot policies require trustworthy evaluation and robot-usable training data, but both are difficult to scale with physical robots alone. Real-robot trials and demonstrations remain the most faithful source of deployment signals, yet they are slow, costly, and hard to reproduce. We present JoyAI-Sim, a simulation-enabled interconversion toolchain for human-robot aligned model evaluation and data generation, denoted as Robot $\rightleftharpoons$ Simulation $\rightleftharpoons$ Human. On the one hand, the Robot $\rightarrow$ Simulation $\rightarrow$ Human pathway supports human-robot aligned model evaluation by reconstructing real-robot tabletop organization tasks as calibrated digital twins for scalable evaluation, while using human embodied feedback to inspect and refine the naturalness of simulated motions. On the other hand, the Human $\rightarrow$ Simulation $\rightarrow$ Robot pathway supports human-robot aligned data generation: it lifts ego-centric human demonstrations into simulation, checks them under robot physical constraints, and converts them into robot-centered trajectories, annotations, and visual observations. Together, these pathways use the JoySim simulator as both a scalable evaluation layer and a physical consistency filter for robot data generation. We further package the core reconstruction, simulation, rendering, and realism-augmentation modules as cloud services on JD Cloud, turning the system into a reusable and scalable infrastructure for robot data generation and model evaluation.

机器人仿真数据生成数字孪生

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。