arXiv:2503.19976cs.GRcs.CV2025-03CVPR被引 3

用神经场+连续物理模型,精准追踪布料等柔性物体的细微形变。

Thin-Shell-SfT: Fine-Grained Monocular Non-rigid 3D Surface Tracking with Neural Deformation Fields

  • 用连续神经场表示表面,避免离散网格误差
  • 结合基尔霍夫-洛夫薄壳物理模型提升形变稳定性
  • 基于3D高斯点云渲染,适合复杂布料细节重建

从单目RGB视频重建高度可变形表面(如布料)是极具挑战的问题,现有方法难以一致且准确地恢复细粒度表面细节。现有技术多依赖统计、神经或物理先验,采用非自适应的离散表面表示(如多边形网格),逐帧优化导致误差累积,且网格可微渲染梯度较差。为此,我们提出ThinShell-SfT,一种新的非刚性3D表面追踪方法,将表面建模为隐式连续时空神经场,引入基于基尔霍夫-洛夫模型的连续薄壳物理先验进行空间正则化,显著区别于以往离散化方案。同时,利用3D高斯点云实现可微渲染,基于分析-合成原理优化形变。实验表明,该方法在定性和定量上均优于现有工作,得益于连续表面表达、定制化物理先验与表面驱动的3D高斯表示。

原文摘要 · Abstract (English)

3D reconstruction of highly deformable surfaces (e.g. cloths) from monocular RGB videos is a challenging problem, and no solution provides a consistent and accurate recovery of fine-grained surface details. To account for the ill-posed nature of the setting, existing methods use deformation models with statistical, neural, or physical priors. They also predominantly rely on nonadaptive discrete surface representations (e.g. polygonal meshes), perform frame-by-frame optimisation leading to error propagation, and suffer from poor gradients of the mesh-based differentiable renderers. Consequently, fine surface details such as cloth wrinkles are often not recovered with the desired accuracy. In response to these limitations, we propose ThinShell-SfT, a new method for non-rigid 3D tracking that represents a surface as an implicit and continuous spatiotemporal neural field. We incorporate continuous thin shell physics prior based on the Kirchhoff-Love model for spatial regularisation, which starkly contrasts the discretised alternatives of earlier works. Lastly, we leverage 3D Gaussian splatting to differentiably render the surface into image space and optimise the deformations based on analysis-bysynthesis principles. Our Thin-Shell-SfT outperforms prior works qualitatively and quantitatively thanks to our continuous surface formulation in conjunction with a specially tailored simulation prior and surface-induced 3D Gaussians. See our project page at https://4dqv.mpiinf.mpg.de/ThinShellSfT.

3D重建非刚性跟踪神经场布料模拟

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。