用3D高斯点云重建双手抓取物体，无需预设模板。
1st Place Solution to the 8th HANDS Workshop Challenge -- ARCTIC Track: 3DGS-based Bimanual Category-agnostic Interaction Reconstruction
- 基于3D高斯泼溅，直接建模手与物体的3D形态。
- 引入掩码损失和3D接触损失，提升遮挡下的重建精度。
- 在ARCTIC数据集上达到38.69的CD_h指标，表现领先。
本文介绍我们针对ECCV 2024年第八届HANDS研讨会（ARCTIC赛道）挑战赛的冠军解决方案。该任务要求从单目视频中重建双手与物体的3D形态，且不依赖预定义模板。由于双手与物体在操作过程中存在严重遮挡与动态接触，该任务极具挑战性。我们通过引入掩码损失和3D接触损失分别缓解遮挡问题与接触建模误差，并采用3D高斯泼溅（3DGS）进行重建。最终，在ARCTIC测试集上，我们的方法在主评价指标CD$_h$上取得了38.69的优异成绩。
原文摘要 · Abstract (English)
This report describes our 1st place solution to the 8th HANDS workshop challenge (ARCTIC track) in conjunction with ECCV 2024. In this challenge, we address the task of bimanual category-agnostic hand-object interaction reconstruction, which aims to generate 3D reconstructions of both hands and the object from a monocular video, without relying on predefined templates. This task is particularly challenging due to the significant occlusion and dynamic contact between the hands and the object during bimanual manipulation. We worked to resolve these issues by introducing a mask loss and a 3D contact loss, respectively. Moreover, we applied 3D Gaussian Splatting (3DGS) to this task. As a result, our method achieved a value of 38.69 in the main metric, CD$_h$, on the ARCTIC test set.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。