仅用单视频和合成数据，就能准确还原可动物体的3D结构与关节参数。
sim2art: Accurate Articulated Object Modeling from a Single Video using Synthetic Training Data Only
- 基于每帧点采样+短时光流与语义特征，构建鲁棒表示
- 在真实视频上表现优于现有方法，且无需域适应
- 适合无标注新物体快速建模，可扩展性强
从单目视频中理解可动物体是机器人和数字孪生的关键挑战。现有方法常依赖多视角、高精度扫描或易失效的长期特征追踪。本文提出 sim2art，仅用合成数据训练，即可从自由移动相机拍摄的单视频中恢复物体的3D部件分割与关节参数。核心思路是基于每帧表面点采样，结合短时场景光流与 DINOv3 语义特征，避免对长时对应关系的依赖。该表示在仿真与真实间差异极小，无需领域自适应。同时，仅依赖单视角可见性，使几何表示在噪声与遮挡下仍保持一致性。采用合适的 Transformer 架构，模型在真实序列上表现优异，超越现有优化与跟踪依赖方法。为填补领域基准空白，我们引入两个新数据集，涵盖更多类别与实例。结果表明，sim2art 能有效处理大范围运动与复杂形变，且无需繁琐真实标注，可轻松扩展至新物体类别。
原文摘要 · Abstract (English)
Understanding articulated objects from monocular video is a crucial yet challenging task in robotics and digital twin creation. Existing methods often rely on complex multi-view setups, high-fidelity object scans, or fragile long-term point tracks that frequently fail in casual real-world captures. In this paper, we present sim2art, a data-driven framework that recovers the 3D part segmentation and joint parameters of articulated objects from a single monocular video captured by a freely moving camera. Our core insight is a robust representation based on per-frame surface point sampling, which we augment with short-term scene flow and DINOv3 semantic features. Unlike previous works that depend on error-prone long-term correspondences, our representation is easy to obtain and exhibits a negligible difference between simulation and reality without requiring domain adaptation. Also, by construction, our method relies on single-viewpoint visibility, ensuring that the geometric representation remains consistent across synthetic and real data despite noise and occlusions. Leveraging a suitable Transformer-based architecture, sim2art is trained exclusively on synthetic data yet generalizes strongly to real-world sequences. To address the lack of standardized benchmarks in the field, we introduce two datasets featuring a significantly higher diversity of object categories and instances than prior work. Our evaluations show that sim2art effectively handles large camera motions and complex articulations, outperforming state-of-the-art optimization-based and tracking-dependent methods. sim2art offers a scalable solution that can be easily extended to new object categories without the need for cumbersome real-world annotations. Project webpage: https://aartykov.github.io/sim2art/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。