提出无需依赖问题的视频帧选择方法,提升长视频理解效率与准确性。
MEDR: Query-Independent Frame Selection via Multi-Signal Event Modeling and Dynamic Rescoring

- 通过多信号事件建模整合视觉、运动和文本信息,生成独立于问题的帧集。
- 动态重评分机制提升关键事件覆盖,使固定帧集在多轮对话中保持高精度。
- 无需训练,可重复使用同一帧集,显著提升长视频问答性能。
帧选择是多模态大模型处理长视频的关键环节,受限于视觉令牌和计算预算。均匀采样虽能保持时间覆盖,但可能遗漏短暂出现的重要内容。查询相关的方法虽可提取与问题相关的帧,但每次问答需重新选择,无法共享视觉输入。为此,本文提出无需训练的查询无关帧选择方法 MEDR。其通过多信号事件建模将视觉、运动和文本信号组织为特定时序事件;动态重评分则迭代评估候选帧,依据帧级信号强度、新增事件覆盖和时间邻近性更新得分。最终生成不依赖问题的固定帧集,可跨多个问题复用。在标准基准 Video-MME 上,准确率提升 0.63%-0.89%;在 LongVideoBench 长视频子集上,使用 Qwen3-VL-8B 模型时准确率最高提升 1.23%;整体准确率提升 0.53%,且每个视频仅使用相同帧集应对所有问题。
原文摘要 · Abstract (English)
Frame selection is a fundamental component of multimodal large language models, enabling long videos to be processed under limited visual-token and computational budgets. Uniform sampling preserves temporal coverage but may miss informative content that appears only briefly. To alleviate this limitation, query-dependent methods can retrieve question-relevant frames. However, because the selected frames depend on the current question, the same visual input cannot be directly shared across different questions, and frame selection must be repeated in multi-turn video dialogue. This motivates us to seek a query-independent frame selection method that preserves the reusability of a fixed visual input while improving the coverage of informative events beyond uniform sampling. We propose Multi-Signal Event Modeling and Dynamic Rescoring (MEDR), a training-free and query-independent frame selection method. Multi-Signal Event Modeling organizes complementary visual, motion, and text signals into signal-specific temporal events. Dynamic Rescoring then iteratively reevaluates each candidate relative to the current selected set, updating its score according to frame-level signal strength, additional event coverage, and temporal proximity. The resulting fixed frame set is constructed without observing the query and can be reused across different questions. On the standard benchmark evaluations, MEDR improves model accuracy by 0.63%-0.89% on Video-MME. On the long-video subset of LongVideoBench, it improves accuracy by up to 1.23% with Qwen3-VL-8B. MEDR further improves overall accuracy by 0.53%, while reusing exactly the same frame set for every question about a video.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。