用单次检索提升大模型长视频理解能力，效率更高。
Towards Effective Long Video Understanding of Multimodal Large Language Models via One-shot Clip Retrieval
- 通过单次片段检索增强视频理解，统一分块与跨模态检索
- 在MLVU上使Qwen3-VL 8B性能达GPT-5水平，处理一小时视频<1.2分钟
- 适用于需要高效长视频理解的场景，如智能监控、教育分析
由于内存开销过大，多数多模态大模型（MLLMs）只能处理有限帧数的视频。本文提出一种高效新范式OneClip-RAG，通过单次视频片段检索增强视频理解，在知识完整性和语义连贯性上均优于现有方法。该方法引入新颖的查询引导视频分块算法，将片段分块与跨模态检索统一于一步，避免重复计算。为提升指令遵循能力，我们构建了新数据集SynLongVideo，并设计渐进式训练策略。OneClip-RAG可嵌入三种主流MLLMs，在多个长视频基准上验证。实验表明，其显著提升模型性能，例如将Qwen3-VL 8B在MLVU上的表现提升至GPT-5水平；同时效率优越，可在单张4090显卡上于不到1.2分钟内完成一小时视频的处理。
原文摘要 · Abstract (English)
Due to excessive memory overhead, most Multimodal Large Language Models (MLLMs) can only process videos of limited frames. In this paper, we propose an effective and efficient paradigm to remedy this shortcoming, termed One-shot video-Clip based Retrieval-Augmented Generation (OneClip-RAG). Compared with existing video RAG methods, OneClip-RAG makes full use of the merits of video clips for augmented video understanding in terms of both knowledge integrity and semantic coherence. Besides, it is also equipped with a novel query-guided video chunking algorithm that can unify clip chunking and cross-modal retrieval in one processing step, avoiding redundant computations. To improve instruction following, we further propose a new dataset called SynLongVideo and design a progressive training regime for OneClip-RAG. OneClip-RAG is plugged into three recent MLLMs and validated on a set of long-video benchmarks. Experimental results not only show the obvious performance gains by OneClip-RAG over MLLMs, e.g., boosting Qwen3-VL 8B to the level of GPT-5 on MLVU, but also show its superior efficiency in handling long videos. e.g., enabling LLaVA-Video understand up to an hour of videos in less than 1.2 minutes on a single 4090 GPU.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。