无需点对应，仅用图像信息快速重建变形物体3D形状。
Image-Guided Shape-from-Template Using Mesh Inextensibility Constraints
- 利用图像颜色、梯度和轮廓+网格不可伸展约束进行无监督重建。
- 速度比现有最优方法快400倍，且在遮挡和细节生成上更优。
- 适合实时3D重建任务，尤其适用于复杂遮挡场景。
形状恢复模板（SfT）是一类通过图像/视频重建可变形物体3D形状的方法，通常依赖图像与3D模板纹理间的点对应关系。传统方法在严重遮挡下因对应关系缺失而性能显著下降。现代方法采用无对应方式，借助深度神经网络实现重建，但需大量监督数据。近期研究结合可微物理与图形学，实现完全无监督或自监督重建。本文提出一种仅使用图像观测（颜色特征、梯度、轮廓）和网格不可伸展约束的无监督SfT方法，重建速度比现有最佳无监督方法快400倍。在生成精细细节和处理严重遮挡方面，本方法显著优于现有技术。代码已开源：https://github.com/dvttran/nsft。
原文摘要 · Abstract (English)
Shape-from-Template (SfT) refers to the class of methods that reconstruct the 3D shape of a deforming object from images/videos using a 3D template. Traditional SfT methods require point correspondences between images and the texture of the 3D template in order to reconstruct 3D shapes from images/videos in real time. Their performance severely degrades when encountered with severe occlusions in the images because of the unavailability of correspondences. In contrast, modern SfT methods use a correspondence-free approach by incorporating deep neural networks to reconstruct 3D objects, thus requiring huge amounts of data for supervision. Recent advances use a fully unsupervised or self-supervised approach by combining differentiable physics and graphics to deform 3D template to match input images. In this paper, we propose an unsupervised SfT which uses only image observations: color features, gradients and silhouettes along with a mesh inextensibility constraint to reconstruct at a $400\times$ faster pace than (best-performing) unsupervised SfT. Moreover, when it comes to generating finer details and severe occlusions, our method outperforms the existing methodologies by a large margin. Code is available at https://github.com/dvttran/nsft.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。