用时间曲率指导选帧,高效捕捉长视频关键事件。
CREST: Curvature-Regulated Event-Centric Sampling for Efficient Long-Video Understanding

- 基于帧间相关性的时间曲率动态选帧,无需训练。
- 在固定预算下准确率超越AKS,仅需MIRA 3-4%的预处理成本。
- 适合需要低延迟长视频理解的场景,如实时检索与分析。
从长视频中选择关键帧是组合优化难题。现有方法或依赖启发式策略而缺乏对查询相关时间结构的建模,或采用多阶段检索流程但预处理开销大。本文提出CREST,一种无训练的帧选择方法,其核心思想是利用查询-帧相关性的局部时间几何特性:显著事件附近曲率陡峭,冗余段落则较为平缓。通过局部曲率引导选帧,CREST能在有限帧预算下更有效地分配至短时关键事件与缓慢演进的内容。在固定主干网络和帧预算条件下,CREST在LongVideoBench和VideoMME上优于轻量级基线AKS,同时保持了MIRA(强多阶段检索模型)93–95%的准确率,但仅需其3–4%的预处理成本。在用于诊断时间帧选择的TempRel基准上,相比AKS相对提升6.88%。基于成对LLM评估显示,CREST选帧生成的描述更连贯,两个基准上的胜出率分别为60.58%和54.50%。结果表明,局部时间几何为长视频帧选择提供了简单高效的依据。
原文摘要 · Abstract (English)
Selecting informative frames from long videos is a combinatorial problem that existing methods address either through efficient heuristics without explicit modeling of query-conditioned temporal structure, or through multi stage retrieval pipelines with substantial preprocessing cost. We propose \textbf{CREST}, a training-free frame selection method grounded in the temporal geometry of query--frame relevance. CREST is based on the observation that relevance over time exhibits structured local variation: sharp curvature around salient events and flatter regions in redundant segments. By using local curvature to guide selection, CREST allocates a fixed frame budget more effectively across brief decisive events and slowly evolving evidence. Under a fixed backbone and frame budget, CREST achieves higher accuracy than AKS, a lightweight relevance--coverage baseline, on LongVideoBench and VideoMME, while retaining 93--95\% of the accuracy of MIRA, a stronger multi-stage retrieval pipeline, at only 3--4\% of its preprocessing cost.\footnote{Code and implementation details are included in the supplementary material and will be released publicly upon acceptance.} On TempRel, our diagnostic benchmark for temporal frame selection, CREST achieves a 6.88\% relative improvement over AKS. Pairwise LLM-as-a-judge evaluation further shows that CREST-selected frames yield more coherent frame-conditioned descriptions, with win rates of 60.58\% and 54.50\% on the two benchmarks. These results show that local temporal geometry provides a simple and efficient basis for long-video frame selection.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。