用RGB图像生成新视角，让机械手更灵巧地抓取物体
GRASPLAT: Enabling dexterous grasping through novel view synthesis
- 通过3D高斯点云合成真实抓握场景的新视角图像
- 在真实数据集上提升抓取成功率36.9%（相比现有方法）
- 仅需彩色图像训练，适合实际应用中缺乏3D数据的场景
多指机械手实现灵巧抓取仍是重大挑战。现有方法依赖完整的3D扫描来预测抓取姿态，但在真实场景中获取高质量3D数据困难。本文提出GRASPLAT，一种新颖的抓取框架，仅使用RGB图像进行训练，同时利用一致的3D信息。核心思路是通过合成手抓物体的物理合理图像，回归出对应的成功抓取姿态。我们采用3D高斯点云（3D Gaussian Splatting）生成真实手物交互的高保真新视角图像，实现端到端的RGB数据训练。与以往方法不同，本方法引入光照一致性损失，通过最小化渲染图像与真实图像间的差异来优化抓取预测。我们在合成和真实抓取数据集上进行了大量实验，结果表明，GRASPLAT相比现有基于图像的方法，抓取成功率最高提升36.9%。
原文摘要 · Abstract (English)
Achieving dexterous robotic grasping with multi-fingered hands remains a significant challenge. While existing methods rely on complete 3D scans to predict grasp poses, these approaches face limitations due to the difficulty of acquiring high-quality 3D data in real-world scenarios. In this paper, we introduce GRASPLAT, a novel grasping framework that leverages consistent 3D information while being trained solely on RGB images. Our key insight is that by synthesizing physically plausible images of a hand grasping an object, we can regress the corresponding hand joints for a successful grasp. To achieve this, we utilize 3D Gaussian Splatting to generate high-fidelity novel views of real hand-object interactions, enabling end-to-end training with RGB data. Unlike prior methods, our approach incorporates a photometric loss that refines grasp predictions by minimizing discrepancies between rendered and real images. We conduct extensive experiments on both synthetic and real-world grasping datasets, demonstrating that GRASPLAT improves grasp success rates up to 36.9% over existing image-based methods. Project page: https://mbortolon97.github.io/grasplat/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。