arXiv:2604.25459cs.RO2026-04被引 4

用3D高斯点云加速真实感机器人视觉学习,每秒处理1万帧。

GS-Playground: A High-Throughput Photorealistic Simulator for Vision-Informed Robot Learning

论文配图:GS-Playground: A High-Throughput Photorealistic Simulator for Vision-Informed Robot Learning
图 1 · 摘自论文原文
  • 结合3D高斯点云渲染与并行物理引擎,实现高速高保真仿真。
  • 在640x480分辨率下达到10⁴帧/秒的惊人吞吐率。
  • 自动化重建真实场景,适合视觉感知与复杂操作任务研究。

具身智能研究正转向以视觉为中心的感知范式。尽管大规模并行模拟器推动了基于本体感觉的运动控制突破,但其在视觉驱动任务中的潜力受限于大规模真实感渲染带来的巨大计算开销。同时,仿真用3D资产创建高度依赖人工建模,且显著的“仿真到现实”物理差距阻碍了接触密集型操作策略的迁移。为此,我们提出GS-Playground,一个面向端到端感知学习的多模态仿真框架。开发新型高性能并行物理引擎,专为集成批量3D高斯点云(3DGS)渲染管线设计,确保高保真同步。系统在640x480分辨率下实现10⁴帧/秒的突破性吞吐率,大幅降低大规模视觉强化学习门槛。此外,引入自动化Real2Sim工作流,可重建真实感、物理一致且内存高效的环境,简化复杂仿真场景生成。在运动、导航和操作任务上的大量实验表明,GS-Playground有效弥合了多种具身任务中的感知与物理鸿沟。

原文摘要 · Abstract (English)

Embodied AI research is undergoing a shift toward vision-centric perceptual paradigms. While massively parallel simulators have catalyzed breakthroughs in proprioception-based locomotion, their potential remains largely untapped for vision-informed tasks due to the prohibitive computational overhead of large-scale photorealistic rendering. Furthermore, the creation of simulation-ready 3D assets heavily relies on labor-intensive manual modeling, while the significant sim-to-real physical gap hinders the transfer of contact-rich manipulation policies. To address these bottlenecks, we propose GS-Playground, a multi-modal simulation framework designed to accelerate end-to-end perceptual learning. We develop a novel high-performance parallel physics engine, specifically designed to integrate with a batch 3D Gaussian Splatting (3DGS) rendering pipeline to ensure high-fidelity synchronization. Our system achieves a breakthrough throughput of 10^4 FPS at 640x480 resolution, significantly lowering the barrier for large-scale visual RL. Additionally, we introduce an automated Real2Sim workflow that reconstructs photorealistic, physically consistent, and memory-efficient environments, streamlining the generation of complex simulation-ready scenes. Extensive experiments on locomotion, navigation, and manipulation demonstrate that GS-Playground effectively bridges the perceptual and physical gaps across diverse embodied tasks. Project homepage: https://gsplayground.github.io.

机器人学习真实感仿真3D高斯

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。