从视频中学习物理一致的3D运动场，无需先验知识。
VeloGauss: Learning Physically Consistent Gaussian Velocity Fields from Videos
- 用物理编码和粒子动力学系统学习高斯粒子速度场。
- 在4个公开数据集上实现最优的新视角插值与未来帧预测效果。
- 适合关注物理建模与视频生成融合的研究者。
本文旨在仅从动态多视角视频中联合建模3D场景的几何、外观与物理信息，不依赖任何物理先验。现有方法通常将物理损失作为软约束或嵌入神经网络进行物理模拟，但难以有效学习复杂运动物理规律。尽管速度场建模具备捕捉真实物理信息的潜力，但因缺乏合适的物理约束，当前方法无法正确学习刚性与非刚性粒子间的交互机制。为此，我们提出VeloGauss，可在无物理先验条件下学习复杂动态3D场景的物理属性。该方法通过引入物理编码与粒子动力学系统，为每个高斯粒子学习速度场，并最终结合全局物理约束以确保场景的物理一致性。在四个公开数据集上的大量实验表明，本方法在新视角插值与未来帧外推任务中均达到最先进性能。
原文摘要 · Abstract (English)
In this paper, we aim to jointly model the geometry, appearance, and physical information of 3D scenes solely from dynamic multi-view videos, without relying on any physical priors. Existing works typically employ physical losses merely as soft constraints or integrate physical simulations into neural networks; however, these approaches often fail to effectively learn complex motion physics. Although modeling velocity fields holds the potential to capture authentic physical information, due to the lack of appropriate physical constraints, current methods are unable to correctly learn the interaction mechanisms between rigid and non-rigid particles. To address this, we propose VeloGauss, designed to learn the physical properties of complex dynamic 3D scenes without physical priors. Our method learns the velocity field for each Gaussian particle by introducing a Physics Code and a Particle Dynamics System, and ultimately incorporates Global Physical Constraints to ensure the physical consistency of the scene. Extensive experiments on four public datasets demonstrate that our method outperforms achieves state-of-the-art performance in both Novel View Interpolation and Future Frame Extrapolation tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。