用缓存机制让多镜头视频角色和场景更一致,支持任意长度生成。
FilmWeaver: Weaving Consistent Multi-Shot Videos with Cache-Guided Autoregressive Diffusion
- 分层缓存设计:用镜头记忆和时序记忆分别保持跨镜头一致性和镜头内连贯性。
- 生成长度可变,多轮交互下仍能保持角色、背景一致性,长视频生成效果更优。
- 适合需要连续叙事的视频创作,如影视短片、动画分镜生成。
当前视频生成模型在单镜头合成上表现良好,但在多镜头视频中难以维持角色与背景的一致性,且难以灵活生成任意长度和镜头数的视频。为此,我们提出 extbf{FilmWeaver},一种可生成任意长度多镜头视频的新框架。该框架采用自回归扩散范式实现长度可变的视频生成。为解决一致性难题,核心思路是将问题解耦为跨镜头一致性和镜头内连贯性。通过双层缓存机制实现:镜头记忆缓存前序镜头的关键帧以维持角色与场景身份,时序记忆则保留当前镜头的帧历史以保证运动流畅。该设计支持多轮用户交互,灵活构建多镜头视频。此外,由于解耦结构,方法还可拓展至多概念注入与视频延长等下游任务。为训练该一致性感知模型,我们还构建了高质量多镜头视频数据集。大量实验表明,该方法在一致性和美学质量指标上均优于现有方法,为更一致、可控、叙事驱动的视频内容生成开辟新可能。
原文摘要 · Abstract (English)
Current video generation models perform well at single-shot synthesis but struggle with multi-shot videos, facing critical challenges in maintaining character and background consistency across shots and flexibly generating videos of arbitrary length and shot count. To address these limitations, we introduce \textbf{FilmWeaver}, a novel framework designed to generate consistent, multi-shot videos of arbitrary length. First, it employs an autoregressive diffusion paradigm to achieve arbitrary-length video generation. To address the challenge of consistency, our key insight is to decouple the problem into inter-shot consistency and intra-shot coherence. We achieve this through a dual-level cache mechanism: a shot memory caches keyframes from preceding shots to maintain character and scene identity, while a temporal memory retains a history of frames from the current shot to ensure smooth, continuous motion. The proposed framework allows for flexible, multi-round user interaction to create multi-shot videos. Furthermore, due to this decoupled design, our method demonstrates high versatility by supporting downstream tasks such as multi-concept injection and video extension. To facilitate the training of our consistency-aware method, we also developed a comprehensive pipeline to construct a high-quality multi-shot video dataset. Extensive experimental results demonstrate that our method surpasses existing approaches on metrics for both consistency and aesthetic quality, opening up new possibilities for creating more consistent, controllable, and narrative-driven video content. Project Page: https://filmweaver.github.io
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。