arXiv:2511.07278cs.CV2025-11被引 21

StreamKV让视频大模型高效回答长视频问题,提升准确率与速度。

StreamKV: Streaming Video Question-Answering with Segment-based KV Cache Retrieval and Compression

  • 按语义动态分段,用摘要向量保留关键信息用于检索。
  • 通过引导提示压缩缓存,仅保留问答所需核心内容。
  • 统一检索与压缩模块,层自适应处理,适合长视频实时问答。

视频大语言模型在视频字幕、搜索和摘要方面展现出巨大潜力,但对长真实视频仍面临挑战。现有方法引入检索机制,从缓存中提取相关键值对以提升效率与准确率,但缓存压缩与检索尚未充分探索。本文提出无需训练的StreamKV框架,无缝集成先进的缓存检索与压缩能力。相比传统均匀分割,StreamKV动态划分视频流为语义片段,更好保留语义信息。针对缓存检索,为每个片段计算摘要向量以保留关键层级信息;针对缓存压缩,引入引导提示捕捉片段内核心语义,确保仅保留最相关信息用于问答。此外,StreamKV将检索与压缩统一于单一模块,以层自适应方式执行,进一步提升流式视频问答效果。在公开的StreamingVQA基准上大量实验表明,StreamKV显著优于现有在线视频大模型,在保持高准确率的同时大幅改善内存效率与计算延迟。代码已开源:https://github.com/sou1p0wer/StreamKV。

原文摘要 · Abstract (English)

Video Large Language Models (Video-LLMs) have demonstrated significant potential in the areas of video captioning, search, and summarization. However, current Video-LLMs still face challenges with long real-world videos. Recent methods have introduced a retrieval mechanism that retrieves query-relevant KV caches for question answering, enhancing the efficiency and accuracy of long real-world videos. However, the compression and retrieval of KV caches are still not fully explored. In this paper, we propose \textbf{StreamKV}, a training-free framework that seamlessly equips Video-LLMs with advanced KV cache retrieval and compression. Compared to previous methods that used uniform partitioning, StreamKV dynamically partitions video streams into semantic segments, which better preserves semantic information. For KV cache retrieval, StreamKV calculates a summary vector for each segment to retain segment-level information essential for retrieval. For KV cache compression, StreamKV introduces a guidance prompt designed to capture the key semantic elements within each segment, ensuring only the most informative KV caches are retained for answering questions. Moreover, StreamKV unifies KV cache retrieval and compression within a single module, performing both in a layer-adaptive manner, thereby further improving the effectiveness of streaming video question answering. Extensive experiments on public StreamingVQA benchmarks demonstrate that StreamKV significantly outperforms existing Online Video-LLMs, achieving superior accuracy while substantially improving both memory efficiency and computational latency. The code has been released at https://github.com/sou1p0wer/StreamKV.

视频问答缓存压缩流式处理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。