用文本控制3D高斯点云动画,让静态场景动起来
Gaussians-to-Life: Text-Driven Animation of 3D Gaussian Splatting Scenes
- 结合视频扩散模型与3D运动迁移,实现2D视频到3D的自然映射
- 支持任意物体类别、复杂动作,多视角保持一致性
- 适合想快速生成沉浸式3D动画的创作者和游戏开发者
当前先进的新视角合成方法在静态3D场景的多视角重建上表现优异,但重建结果仍缺乏
原文摘要 · Abstract (English)
State-of-the-art novel view synthesis methods achieve impressive results for multi-view captures of static 3D scenes. However, the reconstructed scenes still lack "liveliness," a key component for creating engaging 3D experiences. Recently, novel video diffusion models generate realistic videos with complex motion and enable animations of 2D images, however they cannot naively be used to animate 3D scenes as they lack multi-view consistency. To breathe life into the static world, we propose Gaussians2Life, a method for animating parts of high-quality 3D scenes in a Gaussian Splatting representation. Our key idea is to leverage powerful video diffusion models as the generative component of our model and to combine these with a robust technique to lift 2D videos into meaningful 3D motion. We find that, in contrast to prior work, this enables realistic animations of complex, pre-existing 3D scenes and further enables the animation of a large variety of object classes, while related work is mostly focused on prior-based character animation, or single 3D objects. Our model enables the creation of consistent, immersive 3D experiences for arbitrary scenes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。