提出新方法实现高分辨率视频外扩，内容更丰富且保持一致
Follow-Your-Canvas: Higher-Resolution Video Outpainting with Extensive Content Generation
- 分块生成+无缝拼接，突破显存限制
- 支持从512×512扩展到1152×2048（9倍）
- 适合需要大尺寸视频外扩的创作场景
本文研究高分辨率视频外扩与大范围内容生成。现有方法在大幅外扩时面临生成质量差和显存不足的问题。为此，我们提出基于扩散模型的方法 Follow-Your-Canvas，其核心设计包括：一、将任务分解为多个空间窗口并分块生成，再无缝融合，突破显存限制，可处理任意尺寸视频；二、将源视频及其相对位置信息注入每个窗口生成过程，确保生成内容与源视频空间布局协调。该方法实现了从512×512到1152×2048（9倍）的大规模视频外扩，生成内容丰富、时空一致且视觉效果优良，定量指标优于多种分辨率与尺度设置下的现有方法。代码已开源。
原文摘要 · Abstract (English)
This paper explores higher-resolution video outpainting with extensive content generation. We point out common issues faced by existing methods when attempting to largely outpaint videos: the generation of low-quality content and limitations imposed by GPU memory. To address these challenges, we propose a diffusion-based method called \textit{Follow-Your-Canvas}. It builds upon two core designs. First, instead of employing the common practice of "single-shot" outpainting, we distribute the task across spatial windows and seamlessly merge them. It allows us to outpaint videos of any size and resolution without being constrained by GPU memory. Second, the source video and its relative positional relation are injected into the generation process of each window. It makes the generated spatial layout within each window harmonize with the source video. Coupling with these two designs enables us to generate higher-resolution outpainting videos with rich content while keeping spatial and temporal consistency. Follow-Your-Canvas excels in large-scale video outpainting, e.g., from 512X512 to 1152X2048 (9X), while producing high-quality and aesthetically pleasing results. It achieves the best quantitative results across various resolution and scale setups. The code is released on https://github.com/mayuelala/FollowYourCanvas
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。