arXiv:2508.18860cs.LGcs.CV2025-08被引 2

提出C-Flat++框架,提升持续学习的稳定性和效率。

C-Flat++: Towards a More Efficient and Powerful Framework for Continual Learning

  • 通过促进平坦损失曲面,增强模型对新旧任务的平衡能力。
  • 在多种场景下均显著提升持续学习性能,且更新成本更低。
  • 可无缝集成到主流持续学习方法中,适合研究与工程落地。

持续学习中平衡对新任务的敏感性与对旧知识的稳定性至关重要。近期,尖锐度感知最小化在迁移学习中表现优异,并被引入持续学习以改善记忆保留和学习效率。然而,仅依赖零阶尖锐度可能在某些情况下偏好更尖锐的极小值,导致结果不够鲁棒且可能次优。本文提出持续平坦度(C-Flat),一种专为持续学习设计的促进平坦损失曲面的方法。C-Flat具备即插即用特性,可无修改地集成至现有代码流程。此外,我们构建了一个通用框架,将C-Flat融入所有主要持续学习范式,并与基于损失极小值和基于平坦极小值的持续学习方法进行全面对比。实验表明,C-Flat在广泛设置下持续提升性能。进一步提出C-Flat++,采用选择性平坦度驱动优化,显著降低更新开销。跨多种持续学习方法、数据集和场景的大量实验验证了所提方法的有效性与高效性。代码已开源:https://github.com/WanNaa/C-Flat。

原文摘要 · Abstract (English)

Balancing sensitivity to new tasks and stability for retaining past knowledge is crucial in continual learning (CL). Recently, sharpness-aware minimization has proven effective in transfer learning and has also been adopted in continual learning (CL) to improve memory retention and learning efficiency. However, relying on zeroth-order sharpness alone may favor sharper minima over flatter ones in certain settings, leading to less robust and potentially suboptimal solutions. In this paper, we propose \textbf{C}ontinual \textbf{Flat}ness (\textbf{C-Flat}), a method that promotes flatter loss landscapes tailored for CL. C-Flat offers plug-and-play compatibility, enabling easy integration with minimal modifications to the code pipeline. Besides, we present a general framework that integrates C-Flat into all major CL paradigms and conduct comprehensive comparisons with loss-minima optimizers and flat-minima-based CL methods. Our results show that C-Flat consistently improves performance across a wide range of settings. In addition, we introduce C-Flat++, an efficient yet effective framework that leverages selective flatness-driven promotion, significantly reducing the update cost required by C-Flat. Extensive experiments across multiple CL methods, datasets, and scenarios demonstrate the effectiveness and efficiency of our proposed approaches. Code is available at https://github.com/WanNaa/C-Flat.

持续学习平坦极小值优化器改进

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。