用轻量模块生成答案猜想,再用大模型精准验证,大幅降低长视频推理成本。
TimeProVe: Propose, then Verify for Efficient Long Video Temporal Reasoning in Activities of Daily Living

- 先用轻量模型生成候选答案和证据片段,再针对性调用大模型验证。
- 在真实日常活动数据集上提升7.3%准确率,推理成本降低93%。
- 无需专门训练时间定位,适合高效处理长时视频问答任务。
长视频问答(LVQA)需要在数小时未剪辑的视频中识别稀疏且与查询相关的证据。现有方法要么密集使用大型视觉语言模型(VLMs),计算开销巨大;要么依赖稀疏字幕推理,常遗漏时空定位和运动关键证据。本文提出TimeProVe,一种高效混合框架,实现长视频中的时序定位推理。该框架首先通过轻量模块生成基于动作的答案-证据假设,随后仅对关键片段调用昂贵的VLM进行验证。核心是动作基候选证据(ACE)模块,通过轻量LLM推理将局部动作转化为条件相关候选答案与支持证据窗口。我们还构建了开放性基准OpenTSUBench(OTB),用于评估真实日常活动(ADL)场景下的时序定位推理能力。实验表明,TimeProVe在OTB上优于最强基线7.3%,减少VLM调用75%,推理成本降低93%。此外,无需显式时序定位训练,其在Charades-STA上表现优异,结合定位型VLM后达到当前最优水平。
原文摘要 · Abstract (English)
Long Video Question Answering (LVQA) requires identifying sparse, query-relevant evidence within hours-long untrimmed videos. Existing approaches either process videos densely with large vision-language models (VLMs), incurring prohibitive computational cost, or rely on sparse caption-based reasoning, which often misses temporally localized and motion-centric evidence. We introduce TimeProVe, a cost-efficient hybrid framework for temporally grounded reasoning in long videos. TimeProVe first employs lightweight modules to generate action-grounded answer--evidence hypotheses and subsequently invokes an expensive VLM only for targeted verification. The core of our framework lies in the Action-based Candidate Evidence (ACE) module, which converts temporally localized actions into query-conditioned candidate answers and supporting evidence windows through lightweight LLM reasoning. We further introduce OpenTSUBench (OTB), an open-ended benchmark designed to evaluate temporally grounded reasoning in real-world Activities of Daily Living (ADL) scenarios. Experiments show that TimeProVe outperforms the strongest baseline on OTB by 7.3%, while reducing VLM calls by 75% and inference cost by 93%. Furthermore, without explicit temporal grounding training, TimeProVe achieves competitive performance on Charades-STA, and reaches state-of-the-art results when enhanced with grounding VLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。