arXiv:2505.13997cs.CV2025-05被引 7

提出无需样本的视频增量学习框架,有效防止遗忘并捕捉时空特征。

StPR: Spatiotemporal Preservation and Routing for Exemplar-Free Video Class-Incremental Learning

  • 分离保留时空信息,通过语义通道正则化维持旧知识。
  • 动态路由专家网络,根据时间特征自适应选择模型组件。
  • 无需存储样本或任务编号,适合长期连续学习场景。

视频类增量学习(VCIL)旨在构建能随时间持续学习新动作类别而不遗忘旧知识的模型。与传统类增量学习不同,VCIL需处理时空结构带来的复杂性,难以在避免灾难性遗忘的同时捕捉帧间语义和时间动态。现有方法或依赖样本回放(引发内存与隐私问题),或采用静态图像方法而忽略时序建模。为此,我们提出时空保持与路由框架(StPR),一种统一且无需样本的VCIL方案,显式解耦并保留时空信息。首先,提出帧共享语义蒸馏(FSSD),通过联合考虑语义敏感性和分类贡献,识别出稳定有意义的语义通道,并对其进行选择性正则化以维持先验知识并支持适应。其次,设计基于时间分解的专家混合(TD-MoE),根据时间动态动态路由特定任务专家,实现无需任务ID或存储样本的推理。实验在UCF101、HMDB51和Kinetics400上验证,本方法优于现有基线,同时提升可解释性与效率。代码见补充材料。

原文摘要 · Abstract (English)

Video Class-Incremental Learning (VCIL) seeks to develop models that continuously learn new action categories over time without forgetting previously acquired knowledge. Unlike traditional Class-Incremental Learning (CIL), VCIL introduces the added complexity of spatiotemporal structures, making it particularly challenging to mitigate catastrophic forgetting while effectively capturing both frame-shared semantics and temporal dynamics. Existing approaches either rely on exemplar rehearsal, raising concerns over memory and privacy, or adapt static image-based methods that neglect temporal modeling. To address these limitations, we propose Spatiotemporal Preservation and Routing (StPR), a unified and exemplar-free VCIL framework that explicitly disentangles and preserves spatiotemporal information. First, we introduce Frame-Shared Semantics Distillation (FSSD), which identifies semantically stable and meaningful channels by jointly considering semantic sensitivity and classification contribution. These important semantic channels are selectively regularized to maintain prior knowledge while allowing for adaptation. Second, we design a Temporal Decomposition-based Mixture-of-Experts (TD-MoE), which dynamically routes task-specific experts based on their temporal dynamics, enabling inference without task ID or stored exemplars. Together, StPR effectively leverages spatial semantics and temporal dynamics, achieving a unified, exemplar-free VCIL framework. Extensive experiments on UCF101, HMDB51, and Kinetics400 show that our method outperforms existing baselines while offering improved interpretability and efficiency in VCIL. Code is available in the supplementary materials.

视频增量学习时空建模无样本学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。