arXiv:2607.23951cs.CV2026-07

将视频时间定位从预测两端改为直接预测完整时间段,提升精度。

TimePLE: Rethinking Temporal Representation for Video Temporal Grounding

论文配图:TimePLE: Rethinking Temporal Representation for Video Temporal Grounding
图 1 · 摘自论文原文
  • 直接预测时间区间分布,而非分步预测起点终点
  • 在四个基准上平均mIoU达58.9,短中时长事件提升明显
  • 适用于需要精准时间定位的视频理解任务

视频时间定位(VTG)旨在定位自然语言查询描述的连续视频时段。现有基于视觉语言模型(VLM)的方法通常通过两个端点输出间接生成该区间,端点以离散时间标记或连续边界坐标表示。这些方法在编码方式上不同,但预测目标一致:事件区间仍是推导结果,区间有效性、持续时间及区间级相似性仅被隐式处理。本文提出TimePLE,将VTG从端点预测重构为区间原生定位,直接预测有效时间区间的联合分布。TimePLE将每个区间映射到标准位置-持续时间空间中的一个点,支持点对应有效片段,邻近点代表几何相似区间。给定视频与查询,VLM生成单一隐状态<|TIMESPAN|> token,解码为联合区间分布,经持续时间感知坐标校正后转为连续边界。相同区间表示用于编码输入时间锚点,对齐视频侧时间证据与输出侧区间预测。为可靠对齐隐式区间表示与完整事件区间,我们构建了90K规模的标注样本,并人工验证3K规模基准标注。在四个VTG基准上的实验表明,TimePLE持续优于端点预测基线,平均mIoU达58.9,短/中持续时间事件提升显著。

原文摘要 · Abstract (English)

Video temporal grounding (VTG) aims to localize the continuous video interval described by a natural-language query. However, current VLM-based methods typically produce this interval indirectly through two endpoint outputs, represented either as discrete timestamp tokens or continuous boundary coordinates. These formulations differ in how endpoints are encoded, but not in what is predicted: the event interval remains a derived object, while interval validity, duration, and interval-level similarity are handled only implicitly. We propose TimePLE, which reformulates VTG from endpoint prediction to interval-native grounding by predicting a single joint distribution over valid temporal intervals. TimePLE maps each interval to a point in a canonical position-duration square, where every support point corresponds to a valid span and neighboring points represent geometrically similar intervals. Given a video and query, the VLM generates a single latent <|TIMESPAN|> token whose hidden state is decoded into a joint interval distribution, refined through duration-aware coordinate correction, and converted into continuous boundaries. The same interval representation is used to encode input temporal anchors, aligning video-side temporal evidence with output-side span prediction. To reliably align the latent span representation with complete event intervals, we curate 90K-scale grounded samples and human-verify 3K-scale benchmark annotations. Experiments across four VTG benchmarks show that TimePLE consistently outperforms endpoint prediction baselines, achieving an average mIoU of 58.9, with clear gains on short-duration and medium-duration events.

视频定位时间建模多模态

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。