arXiv:2608.08315cs.CVcs.MM2026-08

让视觉语言模型不训练就能准确定位事件发生时间

Your VLM Already Knows When: Training-Free Temporal Grounding by Asking Yes or No

  • 用二元问答替代直接预测时间,通过第一词概率排序定位
  • 在Charades-STA上准确率从3.8%提升至56.8%,超越多数训练模型
  • 无需任何时间标注数据,适合希望快速部署的开发者

现有的多模态大模型虽能可靠识别事件,却难以判断事件发生的时间。在提示输出时间戳时,强模型在Charades-STA上的[email protected]仅达3.8%,且77%到80%的错误预测具有低输出熵:模型自信出错,基于熵的错误检测性能甚至不如随机分类器。我们发现此失败源于任务接口设计,而非感知能力。固定模型权重,将时间回归改为粗粒度到细粒度的二元问题扫描,仅利用第一个词的概率进行排序,使四个冻结主干网络的[email protected]提升28至50个百分点。剩余误差可分解为两个可测量维度:随主干变化的感知轴,和由输出窗口与事件宽度比值决定的几何轴。基于此分析构建的FV-Action方法,在不使用任何时间监督的情况下,达到Charades-STA上56.8%的[email protected],超过同主干原生管道及最强无训练结果;在TACoS上零样本表现超越所有训练过的视频时间定位模型,并优于直接预测在ActivityNet Captions和QVHighlights上的表现。

原文摘要 · Abstract (English)

Multimodal LLMs that recognise events reliably still fail to say when they happen. Prompted for timestamps, strong VLMs reach as little as $3.8\%$ [email protected] on Charades-STA, and $77$ to $80\%$ of their wrong predictions carry low output entropy: the models are confidently wrong, and entropy-based error detection stays below a random classifier. We show that this failure lives in the task interface, not in perception. Holding the weights fixed, replacing timestamp regression with a coarse-to-fine scan of binary questions, whose first-token probabilities are consumed only as a ranking, raises [email protected] by $28$ to $50$ points across four frozen backbones. The residual failures decompose into two measurable axes: a perception axis that moves with the backbone, and a geometry axis that is analytically predictable from the ratio of the output-window and event widths. FV-Action, the training-free method built on this analysis, reaches $56.8\%$ [email protected] on Charades-STA, above the same backbone's native grounding pipeline and the strongest training-free result on this benchmark; it surpasses every TVG-trained model evaluated zero-shot on TACoS, and improves over direct prediction on ActivityNet Captions and QVHighlights, with no temporal supervision at any stage.

时间定位无训练视觉语言模型二元问答

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。