从视频重建可交互的物理级虚拟孪生体，支持复杂柔性物体实时模拟。
PhysTwin: Physics-Informed Reconstruction and Simulation of Deformable Objects from Videos
- 融合弹簧-质量模型与高斯点云，实现物理与视觉双重逼真
- 仅需稀疏视频即可还原完整几何与密集物理属性
- 适合机器人规划与沉浸式内容创作场景
构建真实物体的物理数字孪生体在机器人、内容生成和扩展现实领域具有巨大潜力。本文提出PhysTwin，一种新颖框架，利用动态物体在交互下的稀疏视频，生成兼具照片级真实感与物理真实性的实时可交互虚拟副本。方法核心包含两部分：(1) 物理信息表示，结合弹簧-质量模型实现真实物理仿真，生成式形状模型建模几何，高斯点云实现渲染；(2) 创新的多阶段优化逆建模框架，从视频中重建完整几何、推断密集物理属性并复现真实外观。该方法融合逆物理框架与视觉感知线索，即使在部分遮挡和有限视角下也能实现高保真重建。PhysTwin可建模绳索、毛绒玩具、布料及快递包裹等各类柔性物体。实验表明，其在重建精度、渲染质量、未来预测及新交互下的仿真表现上均优于现有方法。进一步验证了其在实时交互模拟与基于模型的机器人运动规划中的应用价值。
原文摘要 · Abstract (English)
Creating a physical digital twin of a real-world object has immense potential in robotics, content creation, and XR. In this paper, we present PhysTwin, a novel framework that uses sparse videos of dynamic objects under interaction to produce a photo- and physically realistic, real-time interactive virtual replica. Our approach centers on two key components: (1) a physics-informed representation that combines spring-mass models for realistic physical simulation, generative shape models for geometry, and Gaussian splats for rendering; and (2) a novel multi-stage, optimization-based inverse modeling framework that reconstructs complete geometry, infers dense physical properties, and replicates realistic appearance from videos. Our method integrates an inverse physics framework with visual perception cues, enabling high-fidelity reconstruction even from partial, occluded, and limited viewpoints. PhysTwin supports modeling various deformable objects, including ropes, stuffed animals, cloth, and delivery packages. Experiments show that PhysTwin outperforms competing methods in reconstruction, rendering, future prediction, and simulation under novel interactions. We further demonstrate its applications in interactive real-time simulation and model-based robotic motion planning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。