分两阶段生成全景再插值，实现长距离视角一致的360°视图合成。
Look Beyond: Two-Stage Scene View Generation via Panorama and Video Diffusion
- 先生成全景图再插值，用关键帧锚定视图一致性
- 在环形轨迹中仍保持视角与场景全局一致
- 支持灵活相机控制，适合虚拟漫游等应用
单图像新视角合成因大量未观测区域而高度病态，尤其当目标视角与输入差异显著时。现有方法虽注重源图与生成图间的一致性，却难以保证长距离或环形轨迹下的视图连贯性与正确对齐。本文提出一种两阶段方法：先通过全景扩散模型从输入视角图像学习场景先验，生成360度全景；再从中采样并扭曲得到透视关键帧，作为预训练视频扩散模型的锚点，利用空间噪声扩散过程生成新视角。该设计通过全景引导的关键帧约束，确保长期视图与场景一致性。在多个场景数据集上的实验表明，本方法在用户定义轨迹上生成的视图更连贯，优于现有方法。代码已开源：https://github.com/YiGuYT/LookBeyond。
原文摘要 · Abstract (English)
Novel view synthesis (NVS) from a single image is highly ill-posed due to large unobserved regions, especially for views that deviate significantly from the input. While existing methods focus on consistency between the source and generated views, they often fail to maintain coherence and correct view alignment across long-range or looped trajectories. We propose a model that addresses this by decomposing single-view NVS into a 360-degree scene extrapolation followed by novel view interpolation. This design ensures long-term view and scene consistency by conditioning on keyframes extracted and warped from a generated panoramic representation. In the first stage, a panorama diffusion model learns the scene prior from the input perspective image. Perspective keyframes are then sampled and warped from the panorama and used as anchor frames in a pre-trained video diffusion model, which generates novel views through a proposed spatial noise diffusion process. Compared to prior work, our method produces globally consistent novel views -- even in loop closure scenarios -- while enabling flexible camera control. Experiments on diverse scene datasets demonstrate that our approach outperforms existing methods in generating coherent views along user-defined trajectories. Our implementation is available at https://github.com/YiGuYT/LookBeyond.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。