用计算机生成视频提升视频合成的物理真实感
Synthetic Video Enhances Physical Fidelity in Video Synthesis
- 用电脑渲染视频保持3D一致性，提供真实物理约束
- 实验显示该方法显著减少生成视频中的异常现象
- 适合关注视频生成真实感的研究者和开发者
我们研究如何通过利用计算机图形学管道生成的合成视频来提升视频生成模型的物理真实性。这些渲染视频遵循真实世界的物理规律，如保持3D一致性，可作为宝贵资源以改善视频生成模型。为挖掘其潜力，我们提出一种数据筛选与融合方案，并引入方法将物理真实性迁移至模型，显著减少不必要的伪影。在三个强调物理一致性的代表性任务上进行实验，验证了该方法在提升物理真实性方面的有效性。尽管模型仍缺乏对物理的深层理解，本工作提供了首个实证表明合成视频能增强视频合成的物理真实性。
原文摘要 · Abstract (English)
We investigate how to enhance the physical fidelity of video generation models by leveraging synthetic videos derived from computer graphics pipelines. These rendered videos respect real-world physics, such as maintaining 3D consistency, and serve as a valuable resource that can potentially improve video generation models. To harness this potential, we propose a solution that curates and integrates synthetic data while introducing a method to transfer its physical realism to the model, significantly reducing unwanted artifacts. Through experiments on three representative tasks emphasizing physical consistency, we demonstrate its efficacy in enhancing physical fidelity. While our model still lacks a deep understanding of physics, our work offers one of the first empirical demonstrations that synthetic video enhances physical fidelity in video synthesis. Website: https://kevinz8866.github.io/simulation/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。