从多视角视频中无模板重建动态物体的4D形态，无需光照控制
4DPV: 4D Pet from Videos by Coarse-to-Fine Non-Rigid Radiance Fields
- 分阶段神经变形建模，联合估计相机位姿与4D动态结构
- 在真实复杂形变场景下实现高精度重建，优于现有方法
- 适合关注非刚性物体3D重建的研究者和开发者
我们提出一种自监督的粗到精神经变形模型，仅需多个野外采集的RGB视频序列，即可同时恢复未知物体的相机位姿与4D动态结构。该方法无需预设3D模板、3D训练数据或受控光照条件。模型结合规范空间与图像可变空间，引入具有时空一致性的神经局部二次模型，编码精细细节，并通过规范嵌入建立跨序列对应关系。我们在包含复杂真实形变的挑战性场景中进行全面验证，提供定量评估、定性对比、消融实验及与主流方法的比较。项目代码已开源：https://github.com/smontode24/4DPV。
原文摘要 · Abstract (English)
We present a coarse-to-fine neural deformation model to simultaneously recover the camera pose and the 4D reconstruction of an unknown object from multiple RGB sequences in the wild. To that end, our approach does not consider any pre-built 3D template nor 3D training data as well as controlled illumination conditions, and can sort out the problem in a self-supervised manner. Our model exploits canonical and image-variant spaces where both coarse and fine components are considered. We introduce a neural local quadratic model with spatio-temporal consistency to encode fine details that is combined with canonical embeddings in order to establish correspondences across sequences. We thoroughly validate the method on challenging scenarios with complex and real-world deformations, providing both quantitative and qualitative evaluations, an ablation study and a comparison with respect to competing approaches. Our project is available at https://github.com/smontode24/4DPV.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。