用单段视频生成逼真机器人仿真环境，开源工具免去建模烦恼。
Robotic Learning in your Backyard: A Neural Simulator from Open Source Components
- 基于3D高斯泼溅技术，从单个视频构建真实感虚拟场景。
- 支持自车视角生成、碰撞检测与物体修复，完整覆盖训练需求。
- 适合想快速搭建视觉导航训练环境的研究者和开发者。
3D高斯泼溅技术实现了快速且高质量的新视角合成，为从视频构建照片级真实感仿真环境提供了可能。尽管已有研究验证了该方法的有效性，但相关软件工具仍多为专有或不可获取。本文提出SplatGym，一个用于训练数据驱动机器人控制策略的开源神经仿真器。该系统仅需一段视频即可生成逼真虚拟环境，支持自车视角生成、碰撞检测及虚拟物体补全。我们通过强化学习成功训练了多个视觉导航策略。SplatGym标志着向开源通用神经环境迈出的重要一步，通过提供便捷无限制的工具，大幅拓展了强化学习的应用范围，同时避免了传统3D环境的手动开发。
原文摘要 · Abstract (English)
The emergence of 3D Gaussian Splatting for fast and high-quality novel view synthesize has opened up the possibility to construct photo-realistic simulations from video for robotic reinforcement learning. While the approach has been demonstrated in several research papers, the software tools used to build such a simulator remain unavailable or proprietary. We present SplatGym, an open source neural simulator for training data-driven robotic control policies. The simulator creates a photorealistic virtual environment from a single video. It supports ego camera view generation, collision detection, and virtual object in-painting. We demonstrate training several visual navigation policies via reinforcement learning. SplatGym represents a notable first step towards an open-source general-purpose neural environment for robotic learning. It broadens the range of applications that can effectively utilise reinforcement learning by providing convenient and unrestricted tooling, and by eliminating the need for the manual development of conventional 3D environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。