arXiv:2602.03473cs.LGcs.CV2026-02中稿 · ICML被引 2

提出可扩展到300+任务的持续学习框架,解决长期任务中的稳定性与泛化难题。

Scaling Continual Learning to 300+ Tasks with Bi-Level Routing Mixture-of-Experts

  • 采用双层路由MoE架构,动态激活任务相关专家以增强特征表示
  • 在100至300个任务序列上实现领先性能,超越所有基线方法
  • 发布OmniBenchmark-1K数据集,推动长序列持续学习研究

基于预训练模型的持续学习(尤其是类增量学习,CIL)近年受到广泛关注。然而,在极长任务序列中同时保持特征判别性与全面性、兼顾稳定与可塑性仍是开放问题。本文提出可扩展的持续学习框架CaRE,核心为双层路由混合专家(BR-MoE):第一层动态激活相关任务专用路由器,第二层动态选择并聚合专家,将判别性和全面性特征注入每个中间网络层。同时,构建挑战性数据集OmniBenchmark-1K,用于评估数百任务场景下的CIL性能。大量实验表明,CaRE在多种数据集和任务设置下表现领先,包括经典的5-20任务设置。据我们所知,它是首个可扩展至100至300+非重叠任务序列的持续学习模型,且在该序列长度上显著优于所有基线。代码与数据集已开源。

原文摘要 · Abstract (English)

Continual learning, especially class-incremental learning (CIL), on the basis of a pre-trained model (PTM) has garnered substantial research interest in recent years. However, how to effectively learn both discriminative and comprehensive feature representations while maintaining stability and plasticity over very long task sequences remains an open problem. We propose CaRE, a scalable {C}ontinual Le{a}rner with efficient Bi-Level {R}outing Mixture-of-{E}xperts (BR-MoE). The core idea of BR-MoE is a bi-level routing mechanism: a router selection stage that dynamically activates relevant task-specific routers, followed by an expert routing phase that dynamically activates and aggregates experts, aiming to inject discriminative and comprehensive representations into every intermediate network layer. On the other hand, we introduce a challenging dataset, OmniBenchmark-1K, for CIL performance evaluation on very long task sequences with hundreds of tasks. Extensive experiments show that CaRE demonstrates leading performance across a variety of datasets and task settings, including commonly used CIL datasets with classical CIL settings (e.g., 5-20 tasks). To the best of our knowledge, CaRE is the first continual learner that scales to very long task sequences (ranging from 100 to over 300 non-overlapping tasks), while outperforming all baselines by a large margin on such task sequences. We hope that this work will inspire further research into continual learning over extremely long task sequences. Code and dataset are publicly released at https://github.com/LMMMEng/CaRE.

持续学习MoE长序列类增量

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。