仅用单张视频重建布料3D形状与外观，还原更真实动态效果。
SAFT: Shape and Appearance of Fabrics from Template via Differentiable Physical Simulations from Monocular Video
- 通过可微分物理模拟优化布料形变，解决单目视频深度模糊问题。
- 3D重建误差降低2.64倍，每场景耗时30分钟，效果优于最新方法。
- 同时恢复布料表面细节，适合影视动画、虚拟试衣等应用。
三维动态场景重建是计算机视觉中的经典难题。本文提出一种新方法，结合3D几何重建与外观估计，仅需单张单目RGB视频序列，即可实现布料的物理渲染重建。为获得逼真且高质量的形变与渲染效果，系统采用布料物理模拟与可微分渲染技术。本文引入两项新颖正则化项，有效缓解单目视频中的深度歧义问题，提升重建合理性。相比领域内最新方法，本方法将3D重建误差降低2.64倍，单场景运行时间约为30分钟。优化后的运动轨迹足以支持高精度外观估计，成功从单目视频中恢复出清晰的表面细节。
原文摘要 · Abstract (English)
The reconstruction of three-dimensional dynamic scenes is a well-established yet challenging task within the domain of computer vision. In this paper, we propose a novel approach that combines the domains of 3D geometry reconstruction and appearance estimation for physically based rendering and present a system that is able to perform both tasks for fabrics, utilizing only a single monocular RGB video sequence as input. In order to obtain realistic and high-quality deformations and renderings, a physical simulation of the cloth geometry and differentiable rendering are employed. In this paper, we introduce two novel regularization terms for the 3D reconstruction task that improve the plausibility of the reconstruction by addressing the depth ambiguity problem in monocular video. In comparison with the most recent methods in the field, we have reduced the error in the 3D reconstruction by a factor of 2.64 while requiring a medium runtime of 30 min per scene. Furthermore, the optimized motion achieves sufficient quality to perform an appearance estimation of the deforming object, recovering sharp details from this single monocular RGB video.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。