从真实视频生成可交互的高保真物体数字孪生
ArtiTwinSplat: Interactable Digital Twin Reconstruction via Gaussian Splatting from RGB-D videos
- 基于3D高斯点云与无监督运动分析，自动重建物体结构与关节
- 无需建模或标注，直接在真实视频上完成稳定、可查询的数字孪生构建
- 适合需要实物交互的机器人规划与人机协作场景
在非结构化真实环境中部署机器人需要精确且可交互的物体模型。当前大规模构建此类模型仍是系统集成的关键瓶颈。我们提出ArtiTwinSplat，一种直接从RGB-D视频自动构建可动、照片级真实感数字孪生的框架，无需CAD模型、仿真资源或人工标注。方法基于3D高斯点云，保持几何精度与光照真实感，并结合无监督运动分析，仅从观测运动中恢复部件结构与关节运动规律。通过跟踪与优化阶段，实现稳定、可查询的数字孪生，支持实时渲染、视角控制与交互操作。相比以往依赖仿真的方法，ArtiTwinSplat直接作用于真实观测数据，生成的孪生体可立即用于下游机器人规划与学习系统。该方法为可动物体数字孪生构建提供了实用、可扩展的路径，显著降低具身AI与人机协作中物体操作的集成门槛。
原文摘要 · Abstract (English)
Deploying robots in unstructured real-world environments needs accurate, interactive models of the objects. Constructing these models at scale remains a critical bottleneck for robotic system integration. We present ArtiTwinSplat, a framework that automatically constructs articulated, photo-realistic digital twins of objects directly from RGB-D videos, requiring no CAD models, simulation assets, or manual annotations. Our method is built on 3D Gaussian Splatting that preserve geometric fidelity and photometric realism, coupled with an unsupervised articulation discovery pipeline that recovers part structure and joint kinematics from observed motion alone. With tracking and optimization stages our method provides stable, queryable digital twins that support real-time rendering, viewpoint control, and interactive manipulation. Unlike prior methods confined to simulation, ArtiTwinSplat operates directly on real-world observations and produces twins that are immediately usable by downstream robot planning and learning systems. This method offers a practical, scalable pathway toward digital twin construction, lowering the integration barrier for articulated object manipulation in embodied AI and human-robot collaboration contexts.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。