根据问题动态选关键帧，提升长视频问答准确率
ReQuest: Rethinking-based Question-Aware Frame Selection for Long-Form Video QA
- 按问题意图筛选关键帧，减少无效计算
- 在长视频上准确率提升显著，计算开销可控
- 无需修改主模型，适合各类多模态大模型
近期多模态大语言模型（MLLMs）大幅推进了视频理解，但受限于固定输入令牌预算，长视频问答仍具挑战性，均匀采样不利于证据定位。本文提出 ReQuest，一种基于不确定性的、问题自适应的关键帧选择方法，通过选择性计算将问题意图与相关视频内容对齐。ReQuest 包含：(i) 从 MLLM 生成的监督信号中蒸馏出轻量级问题感知选择器；(ii) Re-thinking 路由机制，在模型不确定时触发额外推理，并采用长度自适应准则；(iii) 基于不确定性的自适应非极大值抑制，选择时间上分散的帧，且根据问题难度调整间距。作为即插即用方案，ReQuest 在不修改或微调底层 MLLM 的前提下提升长视频问答性能。在 Video-MME、MLVU 和 LongVideoBench 上的实验表明，其在保持低计算成本的同时实现一致的准确率提升，尤其在中长视频场景中表现突出。
原文摘要 · Abstract (English)
Recent multimodal large language models (MLLMs) have substantially advanced video understanding, yet long-form video QA remains challenging under fixed input token budgets, where uniform sampling can be inefficient for evidence localization. We propose ReQuest , an uncertainty-driven, question-adaptive keyframe selection pipeline that aligns question intent with relevant video content through selective computation. ReQuest integrates (i) a lightweight question-aware selector distilled from MLLM-generated supervision, (ii) Re-thinking Routing that triggers additional inference only when the model is uncertain with a length-adaptive criterion, and (iii) uncertainty-guided adaptive non-maximum suppression that selects temporally diverse frames while adjusting spacing based on question difficulty. As a plug-andplay method, ReQuest improves long-video QA without modifying or fine-tuning the underlying MLLM. Experiments on Video-MME, MLVU, and LongVideoBench demonstrate consistent accuracy gains with competitive computational cost, with particularly strong improvements in medium and long video regimes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。