单图生成可控制的4D动态场景，30秒完成无需迭代优化。
Diff4Splat: Controllable 4D Scene Generation with Latent Dynamic Reconstruction Models
- 用视频潜空间变换器统一生成先验与几何运动约束。
- 输入单图+相机轨迹，30秒内输出带外观、形状、运动的可变形3D高斯场。
- 适合需要高效动态场景生成的研究者与工业应用。
我们提出Diff4Splat，一种从前向方法，从单张图像合成可控且显式的4D场景。该方法将视频扩散模型的生成先验与大规模4D数据集学习到的几何和运动约束相结合。给定单张输入图像、相机轨迹及可选文本提示，Diff4Splat在一次前向传播中直接预测一个可变形的3D高斯场，编码外观、几何和运动信息，无需测试时优化或后处理修正。其核心是一个视频潜空间变换器，增强视频扩散模型以联合捕捉时空依赖关系，并预测随时间变化的3D高斯基元。训练目标涵盖外观保真度、几何准确性与运动一致性，使Diff4Splat能在30秒内合成高质量4D场景。我们在视频生成、新视角合成与几何提取任务中验证了其有效性，结果表明其性能达到或超越基于优化的方法，同时效率显著更高。
原文摘要 · Abstract (English)
We introduce Diff4Splat, a feed-forward method that synthesizes controllable and explicit 4D scenes from a single image. Our approach unifies the generative priors of video diffusion models with geometry and motion constraints learned from large-scale 4D datasets. Given a single input image, a camera trajectory, and an optional text prompt, Diff4Splat directly predicts a deformable 3D Gaussian field that encodes appearance, geometry, and motion, all in a single forward pass, without test-time optimization or post-hoc refinement. At the core of our framework lies a video latent transformer, which augments video diffusion models to jointly capture spatio-temporal dependencies and predict time-varying 3D Gaussian primitives. Training is guided by objectives on appearance fidelity, geometric accuracy, and motion consistency, enabling Diff4Splat to synthesize high-quality 4D scenes in 30 seconds. We demonstrate the effectiveness of Diff4Splat across video generation, novel view synthesis, and geometry extraction, where it matches or surpasses optimization-based methods for dynamic scene synthesis while being significantly more efficient.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。