arXiv:2501.05017cs.CV2025-01被引 2

通过动态选择安全层与分解权重,实现零开销的少样本增量学习。

Continuous Knowledge-Preserving Decomposition with Adaptive Layer Selection for Few-Shot Class-Incremental Learning

  • 用协方差分解权重,分离出冻结与可学子空间。
  • 自动选敏感度低、冗余高的层进行适应,避免遗忘。
  • 无需额外参数或计算量,适合部署场景。

少样本类增量学习(FSCIL)面临保留旧知识与学习新类之间的平衡难题。现有方法或冻结主干网络以防止灾难性遗忘,牺牲灵活性;或添加新模块,带来高成本。这些方法将预训练模型视为黑箱,忽略了两个关键机会:利用层内冗余表征空间,以及根据遗忘敏感度选择性适配层。我们提出CKPD-FSCIL框架,挖掘预训练权重的未充分利用容量,实现卓越的稳定-可塑性平衡,且推理无额外开销。该设计包含两项连续自适应机制:在权重层面,连续知识保全分解利用特征协方差将每个权矩阵拆分为冻结子空间(保护旧知识)和可学习冗余子空间(用于新任务);在层层面,连续自适应层选择基于适配器敏感度比,自动选出冗余能力高、遗忘风险低的层进行适配。仅针对安全、高潜力子空间与层,实现高效适应。每轮后,学习到的适配器合并回原始权重,确保推理时无额外参数或浮点运算。在多个FSCIL基准上的实验表明,本方法在适应能力和知识保留上均持续优于当前最优方案。代码已公开于https://github.com/xiaojieli0903/CKPD-FSCIL。

原文摘要 · Abstract (English)

Few-Shot Class-Incremental Learning (FSCIL) faces a critical challenge: balancing the retention of prior knowledge with the acquisition of new classes. Existing methods either freeze the backbone to prevent catastrophic forgetting, sacrificing plasticity, or add new modules, incurring high costs. These approaches treat pretrained models as black boxes, overlooking two key opportunities to exploit their internal capacity: reusing redundant representational space within layers and selectively adapting layers based on their sensitivity to forgetting. We propose CKPD-FSCIL, a unified framework that unlocks the underutilized capacity of pretrained weights, achieving a superior stability-plasticity balance with zero inference overhead. Our design integrates two continuously adapting mechanisms: At the weight level, a Continuous Knowledge-Preserving Decomposition mechanism uses feature covariance to split each weight matrix into a frozen subspace that safeguards prior knowledge and a learnable, redundant subspace for new tasks. At the layer level, a Continuous Adaptive Layer Selection mechanism leverages an Adapter Sensitivity Ratio to automatically select layers with the highest redundant capacity and lowest forgetting risk for adaptation. By targeting only safe, high-potential subspaces and layers, CKPD-FSCIL enables efficient adaptation. After each session, the learned adapters are merged back into the original weights, ensuring zero additional parameters or FLOPs during inference. Extensive experiments on multiple FSCIL benchmarks demonstrate that our method consistently outperforms state-of-the-art approaches in both adaptability and knowledge retention. The code is available at https://github.com/xiaojieli0903/CKPD-FSCIL.

少样本学习增量学习模型压缩自适应

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。