用运动图显式控制动态场景重建，支持灵活动画与机器人应用。
Motion Blender Gaussian Splatting for Dynamic Scene Reconstruction
- 用运动图+双四元数皮肤法显式表示运动，替代隐式神经编码。
- 在iPhone数据集上达到当前最优，支持新姿态动画与机器人动作预测。
- 适合需要精确运动操控的机器人仿真与视觉规划任务。
高保真动态场景重建中，高斯点阵已展现强大能力。然而现有方法主要依赖隐式运动表示（如神经网络或每高斯参数编码），难以进一步操控重建运动，仅能回放原始动作，限制了其在机器人领域的应用。为此，我们提出运动图显式表示的运动混合高斯点阵（MBGS）框架。通过双四元数皮肤法将运动图连接处的运动传递至各高斯点，结合可学习的权重绘画函数决定每个连接的影响范围。运动图与3D高斯点阵通过可微渲染联合优化。实验表明，MBGS在极具挑战性的iPhone数据集上表现领先，同时在HyperNeRF上也具竞争力。我们展示了该方法在生成新物体姿态、合成真实机器人示范及基于视觉规划预测机器人动作方面的应用潜力。源代码、模型与视频演示见 http://mlzxy.github.io/motion-blender-gs。
原文摘要 · Abstract (English)
Gaussian splatting has emerged as a powerful tool for high-fidelity reconstruction of dynamic scenes. However, existing methods primarily rely on implicit motion representations, such as encoding motions into neural networks or per-Gaussian parameters, which makes it difficult to further manipulate the reconstructed motions. This lack of explicit controllability limits existing methods to replaying recorded motions only, which hinders a wider application in robotics. To address this, we propose Motion Blender Gaussian Splatting (MBGS), a novel framework that uses motion graphs as an explicit and sparse motion representation. The motion of a graph's links is propagated to individual Gaussians via dual quaternion skinning, with learnable weight painting functions that determine the influence of each link. The motion graphs and 3D Gaussians are jointly optimized from input videos via differentiable rendering. Experiments show that MBGS achieves state-of-the-art performance on the highly challenging iPhone dataset while being competitive on HyperNeRF. We demonstrate the application potential of our method in animating novel object poses, synthesizing real robot demonstrations, and predicting robot actions through visual planning. The source code, models, video demonstrations can be found at http://mlzxy.github.io/motion-blender-gs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。