提出自适应采样方法AdaQ，让大模型用更少帧精准理解长视频。
Towards Fast and Effective Long Video Understanding of Multimodal Large Language Models via Adaptive Quasi-Gaussian Sampling
- 将关键帧选择建模为类高斯采样，动态调整采样区间。
- 仅用64帧即使Qwen3-VL-8B平均性能超越GPT4o 15.8%。
- 无需训练、仅需调一个超参数，适合实时长视频应用。
长视频理解对多模态大语言模型（MLLMs）仍是重大挑战，源于计算与内存开销过大。现有关键帧选择方法因硬性采样原则，灵活性差且噪声高。本文将帧选择定义为类高斯采样问题，提出无需训练的自适应方法AdaQ。受高斯分布3-σ法则启发，目标是为不同输入自动优化3-σ区间：局部查询取小区间，全局查询取大区间，实现鲁棒自适应采样。在四种MLLM及三种嵌入模型上验证，结果表明：相比默认模型和当前最优的关键帧方法，AdaQ显著提升性能，例如使用64帧时，Qwen3-VL-8B平均表现超越GPT4o 15.8%；同时具备优异鲁棒性与高效率，仅需设定1个超参数。
原文摘要 · Abstract (English)
Long video understanding remains a daunting challenge for Multimodal Large Language Models (MLLMs) due to the excessive computation and memory footprint. Thus, keyframe selection is often adopted to mitigate this shortcoming, which however still suffers from low flexibility and high noise due to its hard sampling principle. In this paper, we define video frame selection as a problem of Quasi-Gaussian Sampling, and propose an adaptive and training-free approach termed AdaQ. Inspired by the 3-$σ$ rule of Gaussian distribution, the objective of AdaQ is to achieve the optimal 3-$σ$ interval for different examples, i.e., a smaller 3-$σ$ interval for the local query and a larger one for the global query, thereby facilitating robust and adaptive frame sampling. To validate AdaQ, we apply it to four MLLMs with three embedding models. The extensive experimental results not only show its obvious performance gains over the default MLLMs and the SOTA keyframe selection methods, e.g., helping Qwen3-VL-8B outperform GPT4o by 15.8% on average by using only 64 frames, but also confirm its superior robustness and high efficiency for long-video understanding, e.g., only 1 hyper-parameter needs to be set.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。