动态调整模型大小，既防遗忘又省内存。
Grow, Assess, Compress: Adaptive Backbone Scaling for Memory-Efficient Class Incremental Learning
- 通过生长-评估-压缩循环自适应调节模型规模
- 在多个基准上达到顶尖性能，内存减少73%
- 适合资源受限下的持续学习场景
类别增量学习（CIL）面临的核心挑战是：如何在学习新任务所需的可塑性与防止灾难性遗忘所需稳定性之间取得平衡。虽然基于扩展的方法通过添加特定任务参数有效缓解遗忘，但会导致架构无序增长和内存开销。本文提出一种新型动态缩放框架，通过循环的‘生长、评估、压缩’（GRACE）策略自适应管理模型容量。关键在于，我们在骨干网络扩展的基础上引入饱和度评估阶段，判断模型容量的利用程度。该评估使系统能智能决策：选择继续扩展或压缩骨干网络为精简表示，从而避免参数爆炸。实验表明，该方法在多个CIL基准上实现领先性能，相比纯扩展模型内存减少高达73%。
原文摘要 · Abstract (English)
Class Incremental Learning (CIL) poses a fundamental challenge: maintaining a balance between the plasticity required to learn new tasks and the stability needed to prevent catastrophic forgetting. While expansion-based methods effectively mitigate forgetting by adding task-specific parameters, they suffer from uncontrolled architectural growth and memory overhead. In this paper, we propose a novel dynamic scaling framework that adaptively manages model capacity through a cyclic "GRow, Assess, ComprEss" (GRACE) strategy. Crucially, we supplement backbone expansion with a novel saturation assessment phase that evaluates the utilization of the model's capacity. This assessment allows the framework to make informed decisions to either expand the architecture or compress the backbones into a streamlined representation, preventing parameter explosion. Experimental results demonstrate that our approach achieves state-of-the-art performance across multiple CIL benchmarks, while reducing memory footprint by up to a 73% compared to purely expansionist models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。