通过自适应分层记忆压缩，提升视频流理解效率与实时性。
FluxMem: Adaptive Hierarchical Memory for Streaming Video Understanding
- 分两阶段压缩冗余视觉信息：帧间去重与帧内区域合并。
- 在真实场景下实现76.4（StreamingBench）与67.2（OVO-Bench）的顶尖性能。
- 无需训练，自动调节压缩率，适合实时视频应用。
本文提出FluxMem，一种无需训练的高效流式视频理解框架。通过分层两级设计自适应压缩冗余视觉记忆：(1) 时间邻近选择（TAS）模块移除相邻帧间的冗余视觉标记；(2) 空间域整合（SDC）模块将每帧内重复的空间区域合并为紧凑表示。为适应动态场景，TAS与SDC中引入自适应标记压缩机制，根据场景内在统计自动确定压缩率，无需人工调参。大量实验表明，FluxMem在现有在线视频基准上达到新最优结果，在实时设置下于StreamingBench达76.4、OVO-Bench达67.2，同时在OVO-Bench上降低69.9%延迟与34.5%峰值GPU内存占用。此外，其离线性能也保持强劲，在MLVU上达73.1，且仅使用65%的视觉标记。
原文摘要 · Abstract (English)
This paper presents FluxMem, a training-free framework for efficient streaming video understanding. FluxMem adaptively compresses redundant visual memory through a hierarchical, two-stage design: (1) a Temporal Adjacency Selection (TAS) module removes redundant visual tokens across adjacent frames, and (2) a Spatial Domain Consolidation (SDC) module further merges spatially repetitive regions within each frame into compact representations. To adapt effectively to dynamic scenes, we introduce a self-adaptive token compression mechanism in both TAS and SDC, which automatically determines the compression rate based on intrinsic scene statistics rather than manual tuning. Extensive experiments demonstrate that FluxMem achieves new state-of-the-art results on existing online video benchmarks, reaching 76.4 on StreamingBench and 67.2 on OVO-Bench under real-time settings, while reducing latency by 69.9% and peak GPU memory by 34.5% on OVO-Bench. Furthermore, it maintains strong offline performance, achieving 73.1 on MLVU while using 65% fewer visual tokens.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。