arXiv:2605.10543cs.CV2026-05被引 1

让视频生成模型准确理解重叠事件的时间区间,提升时间控制精度。

TIE: Time Interval Encoding for Video Generation over Events

论文配图:TIE: Time Interval Encoding for Video Generation over Events
图 1 · 摘自论文原文
  • 提出时间区间编码TIE,将时间段作为注意力机制中的核心元素。
  • 在OmniEvents数据集上将时间约束满足率从77.34%提升至96.03%。
  • 无需修改模型结构,可直接嵌入现有扩散模型,适合交互式视频生成。

导演风格提示、机器人动作预测和交互式视频代理需要对并发事件进行时间定位——在该场景下,68%的一般视频片段和超过99%的机器人/游戏片段包含重叠事件,但现有多事件生成模型仍基于单一活跃提示假设。现代视频生成模型(如DiT)通过点式位置编码表示时间,导致时间区间与注意力机制存在根本性维度不匹配。本文提出时间区间编码(TIE),一种基于旋转位置编码的间隔感知泛化方法,将时间区间提升为DiT交叉注意力中的第一类原语。TIE遵循两个基本原理:时间可积性(事件需聚合其完整时长的位置证据)与时长不变性(消除对更长区间的偏差)。在统一核函数下,推导出高效的sinc闭式解,保持标准注意力接口,并通过区间积分自然抑制边界噪声。实验表明,TIE在保留基线模型视觉质量的同时显著提升时间可控性:在OmniEvents数据集上,人类验证的时间约束满足率从77.34%提升至96.03%,时间边界误差由0.261秒降至0.073秒,且改善轨迹级时间对齐指标。代码与数据集已开源。

原文摘要 · Abstract (English)

Director-style prompting, robotic action prediction, and interactive video agents demand temporal grounding over concurrent events -- a regime in which 68% of general clips and over 99% of robotics/gameplay clips contain overlapping events, yet existing multi-event generators rest on a single-active-prompt assumption. However, modern video generators, such as Diffusion Transformers (DiT), represent time as discrete points through point-wise positional encodings. This formulation creates a fundamental dimension mismatch: temporally extended intervals and overlapping events are mathematically unrepresentable to the attention mechanism. In this paper, we propose Time Interval Encoding (TIE), a principled, plug-and-play interval-aware generalization of rotary embeddings that elevates time intervals to first-class primitives inside DiT cross-attention. Rather than introducing another heuristic interval embedding, we show that, within RoPE-compatible bilinear attention, TIE is characterized by two basic principles: Temporal Integrability, which requires an event to aggregate positional evidence over its full duration, and Duration Invariance, which removes the trivial bias toward longer intervals. Under a uniform kernel, this characterization yields an efficient closed-form sinc-based solution that preserves the standard attention interface and naturally attenuates boundary noise through interval integration. Empirically, TIE preserves the visual quality of the base DiT model while substantially improving temporal controllability. In our experiments on the OmniEvents dataset, it improves human-verified Temporal Constraint Satisfaction Rate from 77.34% to 96.03% and reduces temporal boundary error from 0.261s to 0.073s, while also improving trajectory-level temporal alignment metrics. The code and dataset are available at https://github.com/MatrixTeam-AI/TIE.

视频生成时间建模扩散模型事件定位

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。