用视频扩散模型提升新视角生成的逼真度，尤其擅长超出观测范围的视角。
Novel View Extrapolation with Video Diffusion Priors
- 利用SVD生成先验，优化辐射场渲染出的模糊视角。
- 在真实感和清晰度上显著优于传统方法，跨视角合成效果更自然。
- 无需微调SVD，适用于单图或视频输入，高效实用。
新视角合成领域因辐射场方法的发展取得了显著进展，但多数辐射场技术在视角外推方面表现不佳，难以生成远超训练视角的新视图。本文提出ViewExtrapolator，一种利用Stable Video Diffusion（SVD）生成先验的新视角合成方法，通过重构SVD去噪过程，有效修正辐射场渲染中常见的伪影，大幅提升合成视角的清晰度与真实感。该方法为通用型新视角外推框架，可适配点云等不同3D渲染输出，即使仅提供单视角或单视频也可工作。此外，ViewExtrapolator无需微调SVD，具备数据与计算双重高效性。大量实验表明其在新视角外推任务中具有显著优势。
原文摘要 · Abstract (English)
The field of novel view synthesis has made significant strides thanks to the development of radiance field methods. However, most radiance field techniques are far better at novel view interpolation than novel view extrapolation where the synthesis novel views are far beyond the observed training views. We design ViewExtrapolator, a novel view synthesis approach that leverages the generative priors of Stable Video Diffusion (SVD) for realistic novel view extrapolation. By redesigning the SVD denoising process, ViewExtrapolator refines the artifact-prone views rendered by radiance fields, greatly enhancing the clarity and realism of the synthesized novel views. ViewExtrapolator is a generic novel view extrapolator that can work with different types of 3D rendering such as views rendered from point clouds when only a single view or monocular video is available. Additionally, ViewExtrapolator requires no fine-tuning of SVD, making it both data-efficient and computation-efficient. Extensive experiments demonstrate the superiority of ViewExtrapolator in novel view extrapolation. Project page: \url{https://kunhao-liu.github.io/ViewExtrapolator/}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。