用物理仿真生成多样演示，训练快速准确的可变形物体世界模型。
PhysWorld: From Real Videos to World Models of Deformable Objects via Physics-Aware Demonstration Synthesis
- 通过物理仿真构建数字孪生，自适应优化材料属性
- 合成多样化运动模式，提升模型泛化能力
- 轻量GNN模型推理速度比最新方法快47倍
交互式世界模型对机器人、虚拟现实和增强现实至关重要。然而，从有限的真实视频数据中学习符合物理规律的可变形物体动力学模型仍具挑战性，尤其当物体具有空间变化的物理属性时。为解决数据稀缺问题，我们提出PhysWorld框架，利用模拟器合成物理合理且多样的演示来学习高效的世界模型。首先，在MPM模拟器中通过本构模型选择与全局到局部优化构建物理一致的数字孪生。随后，对物理属性施加部件感知扰动，生成多种运动模式，合成大量多样化演示。最后，利用这些演示训练嵌入物理属性的轻量级GNN世界模型，真实视频可用于进一步优化物理属性。实验表明，PhysWorld能准确快速预测各类可变形物体的未来状态，并在新交互上具有良好泛化性能；其推理速度相较近期最优方法PhysTwin提升47倍。
原文摘要 · Abstract (English)
Interactive world models that simulate object dynamics are crucial for robotics, VR, and AR. However, it remains a significant challenge to learn physics-consistent dynamics models from limited real-world video data, especially for deformable objects with spatially-varying physical properties. To overcome the challenge of data scarcity, we propose PhysWorld, a novel framework that utilizes a simulator to synthesize physically plausible and diverse demonstrations to learn efficient world models. Specifically, we first construct a physics-consistent digital twin within MPM simulator via constitutive model selection and global-to-local optimization of physical properties. Subsequently, we apply part-aware perturbations to the physical properties and generate various motion patterns for the digital twin, synthesizing extensive and diverse demonstrations. Finally, using these demonstrations, we train a lightweight GNN-based world model that is embedded with physical properties. The real video can be used to further refine the physical properties. PhysWorld achieves accurate and fast future predictions for various deformable objects, and also generalizes well to novel interactions. Experiments show that PhysWorld has competitive performance while enabling inference speeds 47 times faster than the recent state-of-the-art method, i.e., PhysTwin.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。