提出DSCache，让视频模型高效处理长视频流。
Decouple and Cache: KV Cache Construction for Streaming Video Understanding
- 分离累计与即时缓存，实时更新不丢信息
- 支持超训练长度的位置推断，准确率提升2.5%
- 无需训练，适配现有模型，适合长视频应用
流式视频理解需在有限内存和计算资源下处理无界视频流，面临两大挑战：持续构建新缓存并淘汰旧缓存；且因无法获取无限长数据，模型必须从短序列学习并泛化至长流。现有流式 VideoVLLM 未能有效扩展至无界流，或仅关注缓存复用，忽视缓存构造影响。本文提出 DSCache，一种无需训练的缓存构造机制，使预训练离线模型适配流式场景。DSCache 维护累积历史缓存，同时按需构建独立的即时缓存，与历史缓存解耦以保留近期输入信息。为支持超出训练长度的位置外推，DSCache 引入位置无关编码策略，确保缓存可适应未见位置，防止位置溢出。在流式视频问答基准测试中，DSCache 达到当前最优性能，平均准确率较前序方法提升 2.5%。
原文摘要 · Abstract (English)
Streaming video understanding requires processing unbounded video streams with limited memory and computation, posing two key challenges. First, continuously constructing new and evicting old key-value(KV) caches is required for unbounded streams. Secondly, due to the high cost of collecting and training on unbounded streams, models must learn from short sequences while generalizing to long streams. Existing streaming VideoVLLMs fail to scale to unbounded video streams or focus on cache reuse strategies, leaving the impact of cache construction underexplored. In this paper, we propose Decoupled Streaming Cache(DSCache), a training-free cache construction mechanism that adapts pretrained offline models to streaming settings. DSCache maintains a cumulative past KV cache while constructing a separate instant cache on-demand, decoupled from past caches to preserve the informativeness of recent inputs. To enable position extrapolation beyond the training length, DSCache further incorporates a position-agnostic encoding strategy, ensuring KV caches to support unseen positions and preventing position overflow. Experiments on Streaming Video QA benchmarks demonstrate DSCache's state-of-the-art performance, with an average 2.5% accuracy gains over prior methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。