arXiv:2504.07940cs.CV2025-04ICCV被引 15

将普通视频转为全景视频,让观看者自由切换视角。

Beyond the Frame: Generating 360 Panoramic Videos from Perspective Videos

  • 用真实全景视频数据训练模型,实现视角转换。
  • 生成的全景视频时空一致,画面自然流畅。
  • 适合虚拟现实、互动视频等场景使用。

360° 视频已成为呈现动态视觉世界的重要媒介。与传统摄像头的“隧道视野”相比,其无边界视场提供了更完整的环境视角。尽管现有视频生成模型在标准视频生成方面表现优异,但生成完整全景视频的能力仍有限。本文研究视频到360°视频的生成任务:给定一个透视视频输入,目标是生成与原视频内容一致的完整全景视频。不同于常规视频生成,输出视场显著扩大,模型需深入理解场景的空间布局和物体运动以保持时空一致性。为此,我们利用网络上丰富的360°视频,构建高质量数据筛选流程,形成成对训练数据;并设计一系列几何与运动感知操作,促进学习过程,提升生成质量。实验表明,模型可从真实场景透视视频生成逼真且连贯的360°视频。此外,展示了其在视频稳定、视角控制及交互式视觉问答中的应用潜力。

原文摘要 · Abstract (English)

360° videos have emerged as a promising medium to represent our dynamic visual world. Compared to the "tunnel vision" of standard cameras, their borderless field of view offers a more complete perspective of our surroundings. While existing video models excel at producing standard videos, their ability to generate full panoramic videos remains elusive. In this paper, we investigate the task of video-to-360° generation: given a perspective video as input, our goal is to generate a full panoramic video that is consistent with the original video. Unlike conventional video generation tasks, the output's field of view is significantly larger, and the model is required to have a deep understanding of both the spatial layout of the scene and the dynamics of objects to maintain spatio-temporal consistency. To address these challenges, we first leverage the abundant 360° videos available online and develop a high-quality data filtering pipeline to curate pairwise training data. We then carefully design a series of geometry- and motion-aware operations to facilitate the learning process and improve the quality of 360° video generation. Experimental results demonstrate that our model can generate realistic and coherent 360° videos from in-the-wild perspective video. In addition, we showcase its potential applications, including video stabilization, camera viewpoint control, and interactive visual question answering.

全景视频视频生成视角转换

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。