arXiv:2608.29958cs.CV2026-08

通过分析相似度曲线形状,精准选择长视频中事件的核心、过渡和背景帧。

RIDGE: Region-Informed Derivative-Guided Evidence Selection for Long Video Understanding

论文配图:RIDGE: Region-Informed Derivative-Guided Evidence Selection for Long Video Understanding
图 1 · 摘自论文原文
  • 将帧-查询相似度序列视为时间信号,按结构分段筛选。
  • 在四个数据集上多数场景优于现有方法,且无需训练或迭代推理。
  • 适合需要高效理解长视频的场景,如自动摘要与问答系统。

长视频包含的视觉内容远超大型视觉语言模型(LVLMs)在固定视觉标记预算下的处理能力,因此帧选择至关重要。现有查询感知选择器通常基于帧-查询相关性评分构建紧凑子集,但大多将相似度序列仅视为待排序或采样的数值,忽视其随时间演变的形状所蕴含的信息:事件的出现、峰值及消退过程。这可能导致关键的解释性、上下文性或后续帧被遗漏,因为它们可能位于相关性峰值的上升或下降阶段,得分较低。本文提出RIDGE,一种将帧-查询相似度曲线视为时间信号的帧选择框架。通过局部变化与曲率分析,RIDGE将时间轴划分为结构化区域,并对不同区域实施针对性选择,以在固定预算下保留事件核心、过渡、铺垫、余波及上下文帧。该方法为预计算的帧-查询分数提供轻量级后处理,无需训练或迭代调用LVLM。在四个长视频基准和三种骨干网络上,RIDGE在多数设置中表现最佳,其余场景亦保持竞争力。

原文摘要 · Abstract (English)

Long videos contain far more visual content than Large Vision-Language Models (LVLMs) can process under a fixed visual-token budget, making frame selection essential. Existing query-aware selectors usually estimate frame-query relevance and build a compact subset from high-scoring frames. Although their mechanisms differ, the similarity sequence is still often treated primarily as values to rank or sample from, rather than as an ordered signal whose shape reflects how query-relevant evidence emerges, peaks, and fades over time. This can obscure frames that explain, contextualize, or follow an event, because such evidence may lie on the rising or falling sides of a nearby relevance peak and receive lower absolute scores. We propose RIDGE, a frame selection framework that reads the frame-query similarity curve as a temporal signal. By using local changes and curvature, RIDGE partitions the timeline into structural regions and applies region-specific selection to preserve event cores, transitions, buildup, aftermath, and contextual frames under a fixed budget. It is a lightweight post-processing step on precomputed frame-query scores and requires neither training nor iterative LVLM calls. Across four long-video benchmarks and three backbones, RIDGE achieves the best performance in most settings and remains competitive in the others.

视频理解帧选择长视频信号处理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。