自动生成真实超市场景,支持智能体长时任务测试
MarketGen: A Scalable Simulation Platform with Auto-Generated Embodied Supermarket Environments
- 基于多模态输入的生成框架,自动构建结构化超市环境
- 提供1100+商品与可参数化设施资产,支持复杂任务模拟
- 涵盖收银员卸货、售货员取货两类长周期任务,适合商业智能体研究
现有机器人数据集和基准主要聚焦于家庭或桌面场景,且任务时长较短,难以支撑复杂商业环境中的具身智能体研发。为此,我们提出MarketGen——一个可扩展的仿真平台,具备自动构建复杂超市环境的能力。该平台采用新型基于智能体的程序化内容生成(PCG)框架,支持文本和参考图像等多种输入方式,并融合真实商业设计原则,实现完整、结构化且逼真的超市场景自动生成。平台配套提供包含1100多个商品及可参数化设施资产的3D资源库。在此基础上,我们构建了一个新型基准,涵盖两类超市日常任务:(1) 收银台卸货(长时程桌面操作任务,针对收银员智能体);(2) 货架间物品收集(复杂移动操作任务,针对售货员智能体)。通过大量实验验证,包括模块化智能体系统部署与成功从仿真到现实的迁移,证明了平台的有效性。MarketGen为复杂商业场景下的具身人工智能研究提供了全面支持。
原文摘要 · Abstract (English)
The development of embodied agents for complex commercial environments is hindered by a critical gap in existing robotics datasets and benchmarks, which primarily focus on household or tabletop settings with short-horizon tasks. To address this limitation, we introduce MarketGen, a scalable simulation platform with automatic scene generation for complex supermarket environments. MarketGen features a novel agent-based Procedural Content Generation (PCG) framework. It uniquely supports multi-modal inputs (text and reference images) and integrates real-world design principles to automatically generate complete, structured, and realistic supermarkets. We also provide an extensive and diverse 3D asset library with a total of 1100+ supermarket goods and parameterized facilities assets. Building on this generative foundation, we propose a novel benchmark for assessing supermarket agents, featuring two daily tasks in a supermarket: (1) Checkout Unloading: long-horizon tabletop tasks for cashier agents, and (2) In-Aisle Item Collection: complex mobile manipulation tasks for salesperson agents. We validate our platform and benchmark through extensive experiments, including the deployment of a modular agent system and successful sim-to-real transfer. MarketGen provides a comprehensive framework to accelerate research in embodied AI for complex commercial applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。