提出一种无需训练的注意力机制，让视频生成更忠实于多事件描述且时间连贯。
TS-Attn: Temporal-wise Separable Attention for Multi-Event Video Generation
- 将注意力按时间分层，动态调整视觉与文本的对齐关系
- 在多个模型上提升33.5%的生成质量评分，仅增加2%推理时间
- 可直接接入现有模型，适合多事件视频生成任务
从包含多个连续动作的复杂时间描述中生成高质量视频是当前未解难题。现有方法面临本质权衡：多次输入短提示可提升动作准确性，但牺牲时间连贯性；单一复杂提示保持连贯性，却弱化对提示的遵循能力。我们归因于两个核心问题：视频内容与提示间的时间错位，以及运动相关视觉对象与其文本条件之间的注意力冲突。为此，我们提出一种无需训练的新型注意力机制——时间可分离注意力（TS-Attn），通过动态重排注意力分布，确保多事件场景下的时间感知与全局一致性。该机制可无缝集成至多种预训练文生视频模型中，在Wan2.1-T2V-14B和Wan2.2-T2V-A14B上分别将StoryEval-Bench得分提升33.5%和16.4%，推理时间仅增加2%。同时支持跨模型的即插即用，适用于多事件图像到视频生成。代码与项目页见https://github.com/Hong-yu-Zhang/TS-Attn。
原文摘要 · Abstract (English)
Generating high-quality videos from complex temporal descriptions that contain multiple sequential actions is a key unsolved problem. Existing methods are constrained by an inherent trade-off: using multiple short prompts fed sequentially into the model improves action fidelity but compromises temporal consistency, while a single complex prompt preserves consistency at the cost of prompt-following capability. We attribute this problem to two primary causes: 1) temporal misalignment between video content and the prompt, and 2) conflicting attention coupling between motion-related visual objects and their associated text conditions. To address these challenges, we propose a novel, training-free attention mechanism, Temporal-wise Separable Attention (TS-Attn), which dynamically rearranges attention distribution to ensure temporal awareness and global coherence in multi-event scenarios. TS-Attn can be seamlessly integrated into various pre-trained text-to-video models, boosting StoryEval-Bench scores by 33.5% and 16.4% on Wan2.1-T2V-14B and Wan2.2-T2V-A14B with only a 2% increase in inference time. It also supports plug-and-play usage across models for multi-event image-to-video generation. The source code and project page are available at https://github.com/Hong-yu-Zhang/TS-Attn.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。