用3D点云记忆实现视频生成的空间一致性增强
Spatia: Video Generation with Updatable Spatial Memory
- 引入3D点云作为持续空间记忆，动态更新
- 通过视觉SLAM迭代优化记忆，提升空间一致性
- 支持相机控制与3D交互编辑，适合长视频生成
现有视频生成模型因视频信号密度高、维度大，难以保持长期时空一致性。为此，我们提出Spatia，一种基于空间记忆的视频生成框架，显式地保留3D场景点云作为持久空间记忆。Spatia通过条件生成视频片段并利用视觉SLAM持续更新该记忆，实现动态-静态解耦，显著提升生成过程中的空间一致性，同时保持生成真实动态实体的能力。此外，Spatia支持显式相机控制与3D感知交互编辑，为可扩展的、基于记忆的视频生成提供几何基础。
原文摘要 · Abstract (English)
Existing video generation models struggle to maintain long-term spatial and temporal consistency due to the dense, high-dimensional nature of video signals. To overcome this limitation, we propose Spatia, a spatial memory-aware video generation framework that explicitly preserves a 3D scene point cloud as persistent spatial memory. Spatia iteratively generates video clips conditioned on this spatial memory and continuously updates it through visual SLAM. This dynamic-static disentanglement design enhances spatial consistency throughout the generation process while preserving the model's ability to produce realistic dynamic entities. Furthermore, Spatia enables applications such as explicit camera control and 3D-aware interactive editing, providing a geometrically grounded framework for scalable, memory-driven video generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。