从单目视频重建可动物体数字孪生,精度提升近百倍。
VideoArtGS: Building Digital Twins of Articulated Objects from Monocular Video
- 用运动先验引导3D轨迹分析,解决单视角下几何与动作混淆问题。
- 提出混合中心-网格部件分配模块,精准捕捉部件运动变形。
- 在多个数据集上实现当前最佳重建效果,适合工业数字孪生应用。
从单目视频构建可动物体的数字孪生是计算机视觉中的关键挑战,需同时从有限视角中恢复物体几何、部件分割和关节参数。单目视频因输入简便且易于扩展而具有吸引力,但仅靠视觉监督难以分离物体几何与部件动态,因相机与部件共同运动导致估计不明确。尽管预训练追踪模型提供的运动先验可缓解该问题,如何有效融合这些先验以学习关节参数仍缺乏探索。为此,我们提出VideoArtGS,一种从单目视频重建高保真数字孪生的新方法。设计运动先验引导流程,分析3D轨迹、滤除噪声,并提供可靠的关节参数初始化。还引入混合中心-网格部件分配模块,用于基于关节的形变场,准确捕捉部件运动。VideoArtGS在关节建模与网格重建上达到当前最优性能,重建误差相比现有方法降低约两个数量级。该方法使从单目视频构建实用数字孪生成为可能,建立了视频驱动可动物体重建的新基准。代码与资源已公开:https://videoartgs.github.io。
原文摘要 · Abstract (English)
Building digital twins of articulated objects from monocular video presents an essential challenge in computer vision, which requires simultaneous reconstruction of object geometry, part segmentation, and articulation parameters from limited viewpoint inputs. Monocular video offers an attractive input format due to its simplicity and scalability; however, it's challenging to disentangle the object geometry and part dynamics with visual supervision alone, as the joint movement of the camera and parts leads to ill-posed estimation. While motion priors from pre-trained tracking models can alleviate the issue, how to effectively integrate them for articulation learning remains largely unexplored. To address this problem, we introduce VideoArtGS, a novel approach that reconstructs high-fidelity digital twins of articulated objects from monocular video. We propose a motion prior guidance pipeline that analyzes 3D tracks, filters noise, and provides reliable initialization of articulation parameters. We also design a hybrid center-grid part assignment module for articulation-based deformation fields that captures accurate part motion. VideoArtGS demonstrates state-of-the-art performance in articulation and mesh reconstruction, reducing the reconstruction error by about two orders of magnitude compared to existing methods. VideoArtGS enables practical digital twin creation from monocular video, establishing a new benchmark for video-based articulated object reconstruction. Our work is made publicly available at: https://videoartgs.github.io.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。