arXiv:2607.00983cs.CV2026-07

从长视频中选出最相关且信息丰富的关键帧,提升理解效率。

QCA: Query- and Content-Aware Keyframe Selection for Long Video Understanding

论文配图:QCA: Query- and Content-Aware Keyframe Selection for Long Video Understanding
图 1 · 摘自论文原文
  • 根据查询和内容差异动态分配关键帧数量
  • 128帧下在LongVideoBench达67.8分,优于GPT-4o的256帧
  • 无需训练,可直接接入现有视频大模型

视频理解常受严重时间冗余困扰,处理密集帧序列既语义低效又计算昂贵。尤其当仅有少数帧与查询相关时,该问题更显著。本文提出查询与内容感知(QCA)关键帧选择框架,能从长视频中选出紧凑而信息丰富的帧集。QCA首先将视频划分为时间片段,通过联合建模查询相关性与内容偏离度,估计各片段的信息贡献,并动态分配关键帧预算。在每个片段内,以最相关的帧为锚点,迭代加入新帧以最大化多样性,同时保持高语义相关性。方法无需额外训练,可无缝集成至现有视频大模型。在多个长视频理解基准上实验表明,所提方法达到领先性能并具备强泛化能力。例如,在LongVideoBench上使用128帧达67.8分,优于GPT-4o使用256帧的66.7分。代码已开源。

原文摘要 · Abstract (English)

Video understanding is often plagued by severe temporal redundancy, where processing dense frame sequences is both semantically inefficient and computationally expensive. This challenge is further amplified when only a small subset of frames is truly relevant to the given query. In this paper, we propose a Query- and Content-Aware (QCA) keyframe selection framework that can select a compact yet information-rich set of frames from long videos. QCA first partitions the video into temporal segments and estimates the information contribution of each segment by jointly modeling query relevance and content deviation, and dynamically allocates keyframe budget to each segment. Within each segment, QCA anchors on the most query-relevant frame and iteratively incorporates additional frames to maximize diversity while maintaining high semantic relevance to the query. Crucially, our method requires no additional training and can be seamlessly integrated into existing Video-LLMs. Extensive experiments across multiple long video understanding benchmarks demonstrate that our proposed approach achieves state-of-the-art performance and has strong generalization ability. For instance, QCA achieves 67.8\% on LongVideoBench using 128 frames, while GPT-4o achieves 66.7\% using 256 frames. Our codes are available in \href{https://github.com/hktk07/QCA}{GitHub}.

视频理解关键帧选择视频大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。