用稀疏模块冻结大部分预训练模型，实现低成本持续学习。
Pruned Adaptation Modules: A Simple yet Strong Baseline for Continual Foundation Models
- 仅保留少量任务特异层，大幅减少可训练参数
- 参数量减少6倍，仍能有效防止灾难性遗忘
- 适合评估新方法是否真正进步的透明基线
持续学习领域已从传统分类增量学习（CIL）转向基于基础模型（FM）的方法，但缺乏对轻量级卷积基线的充分比较。为此，我们提出剪枝适配模块（PAM），在冻结大部分预训练ResNet的基础上，通过稀疏任务特异层实现可扩展的持续适应。PAM将可训练参数减少约5倍，总参数量减少约6倍，显著降低持续更新成本。在多个基准测试中，PAM均能有效缓解灾难性遗忘，优于现有最先进的FM-based CIL方法。该结果表明PAM可作为强而透明的基线，弥合传统与基于基础模型的CIL方法之间的差距，为未来研究提供更准确的进展评估依据。代码地址：https://github.com/ElifCerenGokYildirim/PAM。
原文摘要 · Abstract (English)
The continual learning literature has rapidly shifted from traditional class incremental learning (CIL) techniques to foundation model (FM)-based CIL methods without a clear understanding of how these newer approaches compare to strong, lightweight convolutional baselines. This abrupt transition has created a substantial methodological gap, making it difficult to assess whether recent FM-based CIL progress reflects genuine advances or merely the absence of rigorous baselines. To address this gap, we introduce Pruned Adaptation Modules (PAM), a simple yet effective method that freezes the vast majority of the pre-trained ResNet while enabling scalable continual adaptation through sparse task-specific layers. PAM yields up to a ~5x reduction in trainable parameters and a ~6x reduction in total parameters, significantly reducing the cost of continual updates. Across diverse benchmarks, PAM consistently mitigates catastrophic forgetting and outperforms state-of-the-art FM-based CIL approaches. Our findings position PAM as a strong and transparent baseline that helps bridge the gap between traditional and FM-based CIL, guiding future research for a more accurate assessment of true progress in continual adaptation. The code can be found at: https://github.com/ElifCerenGokYildirim/PAM.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。