从单目视频重建可动物体数字孪生,兼顾几何与运动一致性。
Articulat3D: Reconstructing Articulated Digital Twins From Monocular Videos with Geometric and Motion Constraints
- 用运动先验驱动初始化,通过3D点轨迹发现物体的低维运动结构。
- 联合优化几何与运动约束,实现高精度且时间连贯的三维重建。
- 适合需要真实场景下低成本构建数字孪生的研究与工业应用。
从视觉数据构建高保真可动物体数字孪生仍是核心挑战。现有方法依赖多视角静态状态采集,严重限制实际应用扩展性。本文提出Articulat3D,一种从随意拍摄的单目视频中构建数字孪生的新框架,通过联合施加显式的3D几何与运动约束实现。我们首先提出运动先验驱动初始化,利用3D点轨迹挖掘可动运动的低维结构,通过一组紧凑的运动基来建模场景动态,实现场景软分割为多个刚性运动组。在此基础上,提出几何与运动约束精化模块,通过可学习的运动学原语(包含关节轴、枢轴点及每帧运动缩放系数)强制物理合理的构型,得到几何准确且时间连贯的重建结果。大量实验表明,Articulat3D在合成基准和真实世界随意拍摄的单目视频上均达到当前最优性能,显著提升了在非受控真实条件下数字孪生构建的可行性。
原文摘要 · Abstract (English)
Building high-fidelity digital twins of articulated objects from visual data remains a central challenge. Existing approaches depend on multi-view captures of the object in discrete, static states, which severely constrains their real-world scalability. In this paper, we introduce Articulat3D, a novel framework that constructs such digital twins from casually captured monocular videos by jointly enforcing explicit 3D geometric and motion constraints. We first propose Motion Prior-Driven Initialization, which leverages 3D point tracks to exploit the low-dimensional structure of articulated motion. By modeling scene dynamics with a compact set of motion bases, we facilitate soft decomposition of the scene into multiple rigidly moving groups. Building on this initialization, we introduce Geometric and Motion Constraints Refinement, which enforces physically plausible articulation through learnable kinematic primitives parameterized by a joint axis, a pivot point, and per-frame motion scalars, yielding reconstructions that are both geometrically accurate and temporally coherent. Extensive experiments demonstrate that Articulat3D achieves state-of-the-art performance on synthetic benchmarks and real-world casually captured monocular videos, significantly advancing the feasibility of digital twin creation under uncontrolled real-world conditions. Our project page is available at https://maxwell-zhao.github.io/Articulat3D/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。