让文字生成视频时精准控制物体出现时间，无需重新训练。
TempoControl: Temporal Attention Guidance for Text-to-Video Models
- 用注意力图优化概念出现时机，三原则协同控制
- 支持多对象重排序、动作时序调整和音视频同步生成
- 保持视频质量与多样性，适合需要精确时序的创作场景
近期生成式视频模型已能根据自然语言提示生成高质量视频，但普遍缺乏细粒度的时间控制能力，无法让用户指定视觉元素在生成序列中的出现时机。本文提出TempoControl，一种在推理阶段实现视觉概念时间对齐的方法，无需重训练或额外监督。该方法利用文生视频扩散模型中的交叉注意力图，通过新颖的优化策略引导概念出现时间。具体通过三个互补原则实现：使注意力时序模式与控制信号相关（相关性）、在需要可见时增强注意力强度（幅度）、保持语义一致性（熵）。TempoControl在保持高视频质量与多样性的同时，实现了精确的时间控制。我们在多个应用场景中验证其有效性，包括单个或多个物体的时间重排、动作时序调整以及音频对齐的视频生成。
原文摘要 · Abstract (English)
Recent advances in generative video models have enabled the creation of high-quality videos based on natural language prompts. However, these models frequently lack fine-grained temporal control, meaning they do not allow users to specify when particular visual elements should appear within a generated sequence. In this work, we introduce TempoControl, a method that allows for temporal alignment of visual concepts during inference, without requiring retraining or additional supervision. TempoControl utilizes cross-attention maps, a key component of text-to-video diffusion models, to guide the timing of concepts through a novel optimization approach. Our method steers attention using three complementary principles: aligning its temporal pattern with a control signal (correlation), adjusting its strength where visibility is required (magnitude), and preserving semantic consistency (entropy). TempoControl provides precise temporal control while maintaining high video quality and diversity. We demonstrate its effectiveness across various applications, including temporal reordering of single and multiple objects, action timing, and audio-aligned video generation. Project page: https://shira-schiber.github.io/TempoControl/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。