arXiv:2605.09442cs.CVcs.AI2026-05被引 2

让视频生成在换提示时快速切换语义,同时保持画面连贯性。

SWIFT: Prompt-Adaptive Memory for Efficient Interactive Long Video Generation

论文配图:SWIFT: Prompt-Adaptive Memory for Efficient Interactive Long Video Generation
图 1 · 摘自论文原文
  • 用轻量级缓存注入新语义,避免每次重算历史
  • 动态调整记忆窗口大小,稳定段落用小窗口降开销
  • 适合需要多提示连续生成的长视频场景

流式长视频生成面临持续语义切换的挑战,需自适应记忆以维持视觉演化的连贯性。现有方法依赖在提示边界重建缓存或固定记忆预算,导致冗余计算且难以灵活适应语义变化。问题根源在于缓存的历史视频与提示更新不匹配:记忆需保持视觉连续,而提示切换又要求快速语义适应。为此,我们提出SWIFT(Semantic Windowing and Injection for Flexible Transitions),一种无需训练的多提示长视频生成框架,可在因果视频扩散模型中实现高效语义切换并保持时间连贯性。SWIFT引入轻量级语义注入缓存,增强已有缓存而非从头重建;通过头级语义注入,使每个注意力头按其与当前视频状态的对齐程度接收提示更新;进一步设计自适应动态窗口,根据提示阶段分配时间记忆:切换边界使用大局部上下文,稳定段落采用小窗口以降低平均推理成本。为在压缩局部注意力下保持长程语义一致性,还维护分段级语义锚点,将提示条件下的视频历史总结为紧凑记忆令牌并重新引入。相比当前最先进方法,SWIFT在保持生成质量的同时,在单张H100 GPU上达到22.6 FPS,显著提升多提示长视频生成效率。代码已开源:https://github.com/ShanwenTan/SWIFT。

原文摘要 · Abstract (English)

Streaming long-video generation faces a central challenge in continuous semantic switching, requiring adaptive memory to preserve coherent visual evolution. Current approaches rely on cache rebuilding at prompt boundaries or fixed memory budgets, but they introduce redundant computation and limit flexible semantic adaptation. This limitation arises from a mismatch between cached video history and prompt updates, as memory preserves visual continuity while prompt switches demand rapid semantic adaptation. Motivated by this observation, we present SWIFT, Semantic Windowing and Injection for Flexible Transitions, a training-free framework for multi-prompt long-video generation that enables efficient semantic switching while preserving temporal coherence in causal video diffusion models. SWIFT introduces a lightweight Semantic Injection Cache that augments cached video memory rather than reconstructing it from scratch at every prompt boundary. To avoid uniformly perturbing all attention channels, we further perform head-wise semantic injection, so that each attention head receives a prompt update proportional to its alignment with the current video state. In addition, we introduce an Adaptive Dynamic Window that allocates temporal memory according to prompt phase, using larger local context near switching boundaries and smaller windows during stable segments to reduce average inference cost. To preserve long-range semantic consistency under compressed local attention, we further maintain segment-level semantic anchors that summarize prompt-conditioned video history and reintroduce it as compact memory tokens. Compared with current state-of-the-art methods, SWIFT preserves generation quality while achieving 22.6 FPS on a single H100 GPU, establishing a substantially more efficient solution for multi-prompt long-video generation. Our code is available at https://github.com/ShanwenTan/SWIFT.

视频生成扩散模型长视频

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。