压缩视频理解中的视觉记忆图，提升效率且不降精度。
StreamMeCo: Long-Term Agent Memory Compression for Efficient Streaming Video Understanding
- 按记忆图连通性分治：孤立节点用极值采样，连通节点用权重剪枝。
- 70%内存压缩下，检索速度提升1.87倍，平均准确率还提高1.0%。
- 适合长期视频理解场景，尤其对存储和计算资源敏感的应用。
视觉代理记忆在流式视频理解中表现优异，但长期存储带来巨大内存开销，增加存储与计算成本。为此，我们提出StreamMeCo，一种高效的流式代理记忆压缩框架。基于记忆图的连通性，该方法对孤立节点采用无边极值采样，对连通节点实施有边权重剪枝，剔除冗余记忆节点同时保持精度。此外，引入时间衰减记忆检索机制，进一步缓解压缩带来的性能下降。在三个挑战性基准数据集（M3-Bench-robot、M3-Bench-web、Video-MME-Long）上的实验表明，当记忆图压缩率达70%时，内存检索速度提升1.87倍，平均准确率提升1.0%。代码已开源：https://github.com/Celina-love-sweet/StreamMeCo。
原文摘要 · Abstract (English)
Vision agent memory has shown remarkable effectiveness in streaming video understanding. However, storing such memory for videos incurs substantial memory overhead, leading to high costs in both storage and computation. To address this issue, we propose StreamMeCo, an efficient Stream Agent Memory Compression framework. Specifically, based on the connectivity of the memory graph, StreamMeCo introduces edge-free minmax sampling for the isolated nodes and an edge-aware weight pruning for connected nodes, evicting the redundant memory nodes while maintaining the accuracy. In addition, we introduce a time-decay memory retrieval mechanism to further eliminate the performance degradation caused by memory compression. Extensive experiments on three challenging benchmark datasets (M3-Bench-robot, M3-Bench-web and Video-MME-Long) demonstrate that under 70% memory graph compression, StreamMeCo achieves a 1.87* speedup in memory retrieval while delivering an average accuracy improvement of 1.0%. Our code is available at https://github.com/Celina-love-sweet/StreamMeCo.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。