构建交互式仿真平台，训练家用机器人完成复杂家务任务。
AgentWorld: An Interactive Simulation Platform for Scene Construction and Mobile Robotic Manipulation
- 自动构建包含物理模拟的家居场景，支持轮式与人形移动控制。
- 涵盖从抓取到烹饪的多阶段任务，实现仿真到现实的有效迁移。
- 适合研究家庭机器人技能学习与仿真训练的科研人员使用。
我们提出AgentWorld，一个用于开发家用移动操作能力的交互式仿真平台。该平台结合自动化场景构建（包括布局生成、语义资产放置、视觉材质配置和物理模拟）与双模式遥操作系统（支持轮式底盘和人形行走策略），用于数据采集。由此构建的AgentWorld数据集涵盖了从基础动作（如抓取、推拉）到多阶段任务（如递饮品、加热食物）的多样化任务，覆盖客厅、卧室和厨房场景。通过广泛评测行为克隆、动作分块Transformer、扩散策略及视觉-语言-动作模型等模仿学习方法，验证了该数据集在仿真到现实迁移中的有效性。集成系统为复杂家庭环境中可扩展的机器人技能获取提供了完整解决方案，弥合了仿真训练与真实部署之间的差距。代码与数据集将公开于 https://yizhengzhang1.github.io/agent_world/。
原文摘要 · Abstract (English)
We introduce AgentWorld, an interactive simulation platform for developing household mobile manipulation capabilities. Our platform combines automated scene construction that encompasses layout generation, semantic asset placement, visual material configuration, and physics simulation, with a dual-mode teleoperation system supporting both wheeled bases and humanoid locomotion policies for data collection. The resulting AgentWorld Dataset captures diverse tasks ranging from primitive actions (pick-and-place, push-pull, etc.) to multistage activities (serve drinks, heat up food, etc.) across living rooms, bedrooms, and kitchens. Through extensive benchmarking of imitation learning methods including behavior cloning, action chunking transformers, diffusion policies, and vision-language-action models, we demonstrate the dataset's effectiveness for sim-to-real transfer. The integrated system provides a comprehensive solution for scalable robotic skill acquisition in complex home environments, bridging the gap between simulation-based training and real-world deployment. The code, datasets will be available at https://yizhengzhang1.github.io/agent_world/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。