提出新方法提升非刚性三维重建精度，解决时序与模糊性难题。
Deep Non-rigid Structure-from-Motion Revisited: Canonicalization and Sequence Modeling
- 对每段序列独立进行规范化，避免数据集级规整的偏差
- 融合时间信息与子空间约束，提升动态形变建模能力
- 适用于视频中非刚性物体的高精度3D重建任务
非刚性结构从运动（NRSfM）是经典三维视觉问题，输入2D序列以估计对应3D序列。近年来深度神经网络显著推动了该任务进展，但现有方法仍难以处理固有时序特性及运动模糊性。本文从两个角度重新审视深度NRSfM：(1) 规范化与(2) 序列建模。提出一种简单易实现的逐序列规范化方法，取代以往的逐数据集规范化。在此基础上，设计结合时间信息与子空间约束的序列建模方法。实验验证表明，所提端到端深度NRSfM框架在多个常用数据集上均优于先前方法，具备更强的重建鲁棒性与精度。
原文摘要 · Abstract (English)
Non-Rigid Structure-from-Motion (NRSfM) is a classic 3D vision problem, where a 2D sequence is taken as input to estimate the corresponding 3D sequence. Recently, the deep neural networks have greatly advanced the task of NRSfM. However, existing deep NRSfM methods still have limitations in handling the inherent sequence property and motion ambiguity associated with the NRSfM problem. In this paper, we revisit deep NRSfM from two perspectives to address the limitations of current deep NRSfM methods : (1) canonicalization and (2) sequence modeling. We propose an easy-to-implement per-sequence canonicalization method as opposed to the previous per-dataset canonicalization approaches. With this in mind, we propose a sequence modeling method that combines temporal information and subspace constraint. As a result, we have achieved a more optimal NRSfM reconstruction pipeline compared to previous efforts. The effectiveness of our method is verified by testing the sequence-to-sequence deep NRSfM pipeline with corresponding regularization modules on several commonly used datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。