用双点图表示可变形物体的形状与姿态，实现高精度3D重建。
DualPM: Dual Posed-Canonical Point Maps for 3D Shape and Pose Reconstruction
- 提出双点图：一个映射像素到真实3D位置，另一个映射到初始姿态的规范位置。
- 仅用少量合成数据训练，即可在真实图像上实现先进性能，显著优于此前方法。
- 支持非可视区域恢复，适合四足动物等可变形物体的3D分析任务。
深度学习在几何任务中的成功很大程度上依赖于数据表示的选择。近期，DUSt3R引入视角不变点图的概念，将静态场景的3D重建问题统一为预测此类点图。本文针对可变形物体的3D形状与姿态重建问题，提出双点图（DualPM）：从同一张图像中提取一对点图——一个将像素映射到物体在当前姿态下的3D位置，另一个映射到其静止状态下的规范形态。我们还将点图扩展至非视域重建，以恢复自遮挡部分的完整形状。实验表明，3D重建与姿态估计可归约为双重点图预测。我们聚焦于四足动物建模，证明只需每类1-2个合成3D模型即可训练，并有效泛化至真实图像，性能显著超越已有方法。
原文摘要 · Abstract (English)
The choice of data representation is a key factor in the success of deep learning in geometric tasks. For instance, DUSt3R recently introduced the concept of viewpoint-invariant point maps, generalizing depth prediction and showing that all key problems in the 3D reconstruction of static scenes can be reduced to predicting such point maps. In this paper, we develop an analogous concept for a very different problem: the reconstruction of the 3D shape and pose of deformable objects. To this end, we introduce Dual Point Maps (DualPM), where a pair of point maps is extracted from the same image-one associating pixels to their 3D locations on the object and the other to a canonical version of the object in its rest pose. We also extend point maps to amodal reconstruction to recover the complete shape of the object, even through self-occlusions. We show that 3D reconstruction and 3D pose estimation can be reduced to the prediction of DualPMs. Empirically, we demonstrate that this representation is a suitable target for deep networks to predict. Specifically, we focus on modeling quadrupeds, showing that DualPMs can be trained purely on synthetic 3D data, consisting of one or two models per category, while generalizing effectively to real images. With this approach, we achieve significant improvements over previous methods for the 3D analysis and reconstruction of such objects.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。