视频RAG新方法:按块智能选检索配置,提升长视频问答准确率
Rethinking RAG in Long Videos: What to Retrieve and How to Use It?

- 并行多配置检索+块级自适应重排,动态选择最优检索粒度与模态
- 在新基准V-RAGBench上超越8个现有方法,准确率显著提升
- 适合需要高精度长视频理解的场景,如教育、医疗视频分析
检索增强生成正从文本拓展至长时程第一人称视频,系统需跨多模态和时间粒度选择相关片段。然而当前VideoRAG进展受限于两大缺陷:现有基准允许不依赖视频即可回答问题,掩盖了检索错误;先前方法对每个查询采用单一模态-粒度配置,忽视片段级别的差异性。为此,本文提出V-RAGBench,一个包含⟨查询, 证据片段, 答案⟩三元组的基准,支持对检索与生成环节的独立、真实评估;并提出CARVE方法,通过并行运行多种配置的检索器,结合片段自适应重排,为每个片段选出最优配置。该最优配置在生成阶段随片段传递,形成交错的证据结构,使各片段可使用不同配置。CARVE在多个视频问答任务中优于八个近期VideoRAG基线,生成时所用片段融合多种配置,而非常规查询级单一配置,这是查询级方法无法实现的行为。
原文摘要 · Abstract (English)
Retrieval-augmented generation is moving beyond text into long, egocentric video, where systems must select query-relevant chunks across multiple modalities and temporal granularities. Yet progress in VideoRAG is limited by two gaps: existing benchmarks allow queries to be answered without the video, obscuring retrieval errors, and prior methods apply a single modality-granularity configuration per query, ignoring chunk-level variability. We address both by introducing V-RAGBench, a benchmark of $\langle$query, evidence chunk, answer$\rangle$ triplets that enables faithful, decoupled evaluation of retrieval and generation, and CARVE, a simple method that runs parallel retrievers across configurations and employs chunk-adaptive reranking to identify the winning configuration for each chunk. Each chunk then enters the generator under its winning configuration selected during retrieval, yielding an interleaved evidence form where the chunk-level decision propagates across both stages. CARVE outperforms eight recent VideoRAG baselines, with the chunks supplied to the generator interleaving multiple configurations rather than sharing a single one, a behavior unattainable by query-level methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。