arXiv:2508.05001cs.CVcs.LG2025-08ICCV

用压缩视频码解决视频持续学习的存储难题,实现小内存高效学习。

CRAM: Large-scale Video Continual Learning with Bootstrapped Compression

  • 用视频嵌入码替代原始视频存入缓存,降低存储开销。
  • 在EpicKitchens-100和Kinetics-700上用不到2GB存数千条长视频,性能超越旧方法。
  • 提出在线刷新机制,防止压缩模型遗忘,适合资源受限的持续学习场景。

持续学习(CL)使神经网络能从连续输入流中学习,而非依赖独立同分布采样,从而大幅降低存储需求,提升部署系统对自然分布变化的适应能力。本文聚焦于基于回放的视频持续学习,针对视频数据量大、长视频多、持续流导致记忆缓冲区容量受限的问题,提出使用压缩视觉表示——即存储视频嵌入码而非原始帧,并通过随机采样训练分类器。由于在线训练压缩模型也会遭遇灾难性遗忘,我们设计了代码刷新机制:利用旧模型解压,新模型重新压缩。该方法命名为持续刷新异模记忆(CRAM)。我们在EpicKitchens-100和Kinetics-700等大规模视频数据集上验证,仅用不足2GB存储数千条长视频,且显著优于现有方法。

原文摘要 · Abstract (English)

Continual learning (CL) promises to allow neural networks to learn from continuous streams of inputs, instead of IID (independent and identically distributed) sampling, which requires random access to a full dataset. This would allow for much smaller storage requirements and self-sufficiency of deployed systems that cope with natural distribution shifts, similarly to biological learning. We focus on video CL employing a rehearsal-based approach, which reinforces past samples from a memory buffer. We posit that part of the reason why practical video CL is challenging is the high memory requirements of video, further exacerbated by long-videos and continual streams, which are at odds with the common rehearsal-buffer size constraints. To address this, we propose to use compressed vision, i.e. store video codes (embeddings) instead of raw inputs, and train a video classifier by IID sampling from this rolling buffer. Training a video compressor online (so not depending on any pre-trained networks) means that it is also subject to catastrophic forgetting. We propose a scheme to deal with this forgetting by refreshing video codes, which requires careful decompression with a previous version of the network and recompression with a new one. We name our method Continually Refreshed Amodal Memory (CRAM). We expand current video CL benchmarks to large-scale settings, namely EpicKitchens-100 and Kinetics-700, storing thousands of relatively long videos in under 2 GB, and demonstrate empirically that our video CL method outperforms prior art with a significantly reduced memory footprint.

视频持续学习压缩记忆小内存学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。