提出可精确控制全景视频大范围动态运动的新方法。
PanFlow: Decoupled Motion Control for Panoramic Video Generation
- 将相机旋转与光流解耦，实现对全景动态运动的精细控制。
- 在大型全景视频数据集上训练，显著提升运动保真度和时序一致性。
- 适合虚拟现实、视频编辑等需要复杂运动控制的应用场景。
全景视频生成因在虚拟现实和沉浸式媒体中的应用而受到越来越多关注。然而，现有方法缺乏显式的运动控制能力，难以生成具有大范围复杂运动的场景。我们提出PanFlow，一种利用全景球面特性的新方法，将高度动态的相机旋转与输入光流条件解耦，从而实现对大范围动态运动的更精确控制。我们进一步引入球面噪声扭曲策略，提升全景边界处运动的循环一致性。为支持有效训练，我们构建了一个大规模、富含运动信息的全景视频数据集，包含逐帧姿态和光流标注。我们在多种应用场景中展示了该方法的有效性，包括运动迁移和视频编辑。大量实验表明，PanFlow在运动保真度、视觉质量和时序连贯性方面均显著优于先前方法。代码、数据集和模型已公开于https://github.com/chengzhag/PanFlow。
原文摘要 · Abstract (English)
Panoramic video generation has attracted growing attention due to its applications in virtual reality and immersive media. However, existing methods lack explicit motion control and struggle to generate scenes with large and complex motions. We propose PanFlow, a novel approach that exploits the spherical nature of panoramas to decouple the highly dynamic camera rotation from the input optical flow condition, enabling more precise control over large and dynamic motions. We further introduce a spherical noise warping strategy to promote loop consistency in motion across panorama boundaries. To support effective training, we curate a large-scale, motion-rich panoramic video dataset with frame-level pose and flow annotations. We also showcase the effectiveness of our method in various applications, including motion transfer and video editing. Extensive experiments demonstrate that PanFlow significantly outperforms prior methods in motion fidelity, visual quality, and temporal coherence. Our code, dataset, and models are available at https://github.com/chengzhag/PanFlow.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。