arXiv:2603.18912cs.CV2026-03被引 1

用高斯点云快速重建真实手物交互,无需类别先验。

GHOST: Fast Category-agnostic Hand-Object Interaction Reconstruction from RGB Videos using Gaussian Splatting

  • 用2D高斯点云表示手和物体,实现视图一致的动态重建。
  • 在ARCTIC、HO3D等数据集上达到最先进3D重建精度。
  • 适合需要快速生成逼真手物交互的AR/VR与机器人应用。

从单目RGB视频理解真实的手物交互对AR/VR、机器人及具身AI至关重要。现有方法依赖类别特定模板或计算量大,且常产生物理不一致的3D手物对齐。本文提出GHOST(Gaussian Hand-Object Splatting),一种基于2D高斯点云的快速、无类别先验的手物交互重建框架。GHOST将手和物体表示为密集、视图一致的高斯盘,并引入三项关键创新:(1) 几何先验检索与一致性损失,补全被遮挡的物体区域;(2) 抓取感知对齐,优化手部平移与物体尺度以确保真实接触;(3) 手部感知背景损失,避免惩罚被手遮挡的物体区域。GHOST仅需单个RGB视频即可实现完整、物理一致且可动画化的重建,运行速度比先前无类别先验方法快一个数量级。在ARCTIC、HO3D及野外数据集上的大量实验表明,其在3D重建精度与2D渲染质量上均达当前最优,证明其在真实手物交互建模中兼具高效性与鲁棒性。代码已开源:https://github.com/ATAboukhadra/GHOST。

原文摘要 · Abstract (English)

Understanding realistic hand-object interactions from monocular RGB videos is essential for AR/VR, robotics, and embodied AI. Existing methods rely on category-specific templates or heavy computation, yet still produce physically inconsistent hand-object alignment in 3D. We introduce GHOST (Gaussian Hand-Object Splatting), a fast, category-agnostic framework for reconstructing dynamic hand-object interactions using 2D Gaussian Splatting. GHOST represents both hands and objects as dense, view-consistent Gaussian discs and introduces three key innovations: (1) a geometric-prior retrieval and consistency loss that completes occluded object regions, (2) a grasp-aware alignment that refines hand translations and object scale to ensure realistic contact, and (3) a hand-aware background loss that prevents penalizing hand-occluded object regions. GHOST achieves complete, physically consistent, and animatable reconstructions from a single RGB video while running an order of magnitude faster than prior category-agnostic methods. Extensive experiments on ARCTIC, HO3D, and in-the-wild datasets demonstrate state-of-the-art accuracy in 3D reconstruction and 2D rendering quality, establishing GHOST as an efficient and robust solution for realistic hand-object interaction modeling. Code is available at https://github.com/ATAboukhadra/GHOST.

手物交互高斯溅射3D重建实时渲染

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。