提出无需训练的流式视频理解框架,实现低延迟与内存可控。
StreamingTOM: Streaming Token Compression for Efficient Video Understanding
- 两阶段压缩:帧内选关键视觉标记,动态控制预填充开销。
- 4比特在线量化存储,保持活跃缓存恒定,支持任意长视频流。
- 比SOTA快2倍、内存降一半,适配实时视频应用开发。
与离线处理不同,流式视频视觉语言模型面临因果性与累积性双重约束:因果性禁止访问未来帧,而累积性导致标记数量无界增长,造成效率瓶颈。现有方法仅限制LLM后端kv缓存,未解决前端预填充开销。我们提出StreamingTOM,一种免训练、即插即用的两阶段框架,同时缓解前端与后端瓶颈。因果时序压缩设定每帧固定标记预算,基于相邻帧变化和标记显著性选择关键帧,大幅降低每帧预填充成本,仅处理紧凑视觉标记子集,确保可预测延迟。在线量化记忆以4比特格式存储标记,按需检索并解量化相关组,使活跃kv缓存始终受限于流长。实验表明,该方法实现15.7倍的kv缓存压缩比;相比先前最优(LiveVLM),峰值内存降低1.2倍,首次传输时间(TTFT)提升2倍。StreamingTOM在免训练方法中达到顶尖准确率,在离线基准平均达63.8%,在RVS上取得55.8%准确率与3.7分,证明无需模型重训练即可实现具备有限活跃内存的实时视频理解。
原文摘要 · Abstract (English)
Unlike offline processing, streaming video vision-language models face two fundamental constraints: causality and accumulation. Causality prevents access to future frames that offline methods exploit, while accumulation causes tokens to grow unbounded, creating efficiency bottlenecks. However, existing approaches only regulate post-LLM kv-cache, leaving costly pre-LLM prefill unchanged. We introduce StreamingTOM, a training-free, plug-and-play two-stage framework that addresses both pre-LLM and post-LLM bottlenecks. Causal Temporal Reduction imposes a fixed per-frame budget and selects tokens based on adjacent-frame changes and token saliency, drastically reducing per-frame prefill cost by processing only a compact subset of visual tokens, ensuring predictable latency. Online Quantized Memory stores tokens in 4-bit format, retrieves relevant groups on demand, and dequantizes them, keeping the active kv-cache bounded regardless of stream length. Experiments demonstrate our method achieves $15.7\times$ kv-cache compression ratio; compared to prior SOTA (LiveVLM), it delivers $1.2\times$ lower peak memory and $2\times$ faster TTFT. StreamingTOM achieves state-of-the-art accuracy among training-free methods with an average of $63.8\%$ on offline benchmarks and $55.8\%$ accuracy and $3.7$ score on RVS. These results demonstrate that real-time streaming video understanding with bounded active memory is achievable without model retraining.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。