用高斯点云渲染实现零样本机器人抓取，真实世界成功率86.25%
SplatSim: Zero-Shot Sim2Real Transfer of RGB Manipulation Policies Using Gaussian Splatting
- 用高斯点云替代传统网格，生成逼真仿真图像
- 零样本部署下抓取成功率86.25%，接近真实数据训练的97.5%
- 适合做视觉操控的机器人研究者快速验证算法
基于RGB图像的机器人操控策略在仿真到现实的迁移中仍面临巨大挑战，主要源于合成数据与真实视觉之间的显著域偏移。本文提出SplatSim框架，采用高斯点云作为核心渲染单元，降低视觉域差距。通过在模拟器中以高斯点云取代传统网格表示，该方法在保持仿真可扩展性和低成本的同时，生成高度逼真的合成数据。我们在SplatSim中训练操控策略，并以零样本方式部署至真实世界，平均成功率达86.25%，接近在真实数据上训练的97.5%表现。相关视频见项目主页：https://splatsim.github.io
原文摘要 · Abstract (English)
Sim2Real transfer, particularly for manipulation policies relying on RGB images, remains a critical challenge in robotics due to the significant domain shift between synthetic and real-world visual data. In this paper, we propose SplatSim, a novel framework that leverages Gaussian Splatting as the primary rendering primitive to reduce the Sim2Real gap for RGB-based manipulation policies. By replacing traditional mesh representations with Gaussian Splats in simulators, SplatSim produces highly photorealistic synthetic data while maintaining the scalability and cost-efficiency of simulation. We demonstrate the effectiveness of our framework by training manipulation policies within SplatSim and deploying them in the real world in a zero-shot manner, achieving an average success rate of 86.25%, compared to 97.5% for policies trained on real-world data. Videos can be found on our project page: https://splatsim.github.io
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。