构建100多个真实感3D虚拟世界,助力具身AI研究
UnrealZoo: Enriching Photo-realistic Virtual Worlds for Embodied AI
- 基于虚幻引擎构建超百个高保真虚拟场景
- 环境多样性显著提升强化学习模型泛化能力
- 适合研究多智能体交互与开放世界导航的科研人员
我们提出UnrealZoo,一个基于虚幻引擎构建的超过100个照片级真实感3D虚拟世界集合,旨在反映开放世界环境的复杂性与多样性。提供丰富可玩实体,包括人类、动物、机器人和车辆,适用于具身AI研究。通过优化UnrealCV的API与工具链,支持数据采集、环境增强、分布式训练和基准测试,显著提升渲染与通信效率,支持多智能体交互等高级应用。在视觉导航与追踪任务上的实验表明:1)环境多样性对发展可泛化的强化学习代理具有显著优势;2)当前具身智能体在开放世界中仍面临挑战,包括非结构化地形导航、未见形态适应以及动态物体交互中的闭环控制延迟。UnrealZoo因此成为全面的测试平台,也是推动具身AI向现实部署迈进的重要路径。
原文摘要 · Abstract (English)
We introduce UnrealZoo, a collection of over 100 photo-realistic 3D virtual worlds built on Unreal Engine, designed to reflect the complexity and variability of open-world environments. We also provide a rich variety of playable entities, including humans, animals, robots, and vehicles for embodied AI research. We extend UnrealCV with optimized APIs and tools for data collection, environment augmentation, distributed training, and benchmarking. These improvements achieve significant improvements in the efficiency of rendering and communication, enabling advanced applications such as multi-agent interactions. Our experimental evaluation across visual navigation and tracking tasks reveals two key insights: 1) environmental diversity provides substantial benefits for developing generalizable reinforcement learning (RL) agents, and 2) current embodied agents face persistent challenges in open-world scenarios, including navigation in unstructured terrain, adaptation to unseen morphologies, and managing latency in the close-loop control systems for interacting in highly dynamic objects. UnrealZoo thus serves as both a comprehensive testing ground and a pathway toward developing more capable embodied AI systems for real-world deployment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。