解决长尾分布下增量学习的数值不稳问题,提升小类识别能力。
Spectral-Aware Analytic Class-Incremental Learning for Long-Tailed Distributions
- 提出谱感知正则化框架,针对性修复小类特征空间坍塌
- 在ImageNet-LT和CIFAR-100-LT上准确率提升4.2%~6.8%(绝对值)
- 适合关注高效增量学习与长尾数据场景的研究者
分析型持续学习(ACL)为基于梯度的方法提供了计算高效的替代方案。近期基于递归最小二乘法(RLS)的ACL方法在同类中表现最优,但在长尾分布的类增量学习中显著失效。我们发现,类不平衡会加剧RLS中自相关矩阵的病态性,导致‘尾部’类别出现严重的谱崩溃,其特征子空间在数值上难以区分于噪声。标准岭回归(L2)因采用各向同性正则化,对所有特征方向施加均匀惩罚,无法有效稳定尾部而不会过度压缩头部。为此,我们提出几何-谱修正(GSR),将长尾学习视为谱正则化问题。GSR作为各向异性谱滤波器,通过构建依赖数据的结构化扰动矩阵Δ,选择性放大尾部类别的坍塌特征方向。理论证明,GSR能保证自相关矩阵的稳定秩提升,确保数值稳定性。大量实验表明,GSR在分析型类增量学习中建立新基准,在计算效率与鲁棒泛化之间实现更优平衡。
原文摘要 · Abstract (English)
Analytic Continual Learning (ACL) offers a computationally efficient alternative to gradient-based approaches. Recent ACL methods are based on Recursive Least Squares (RLS) and have achieved the state-of-the-art results compared to other alternatives. However, they falter significantly in Class-Incremental Learning scenarios characterized by Long-Tailed distributions. While the ill-conditioning of the autocorrelation (Gram) matrix is a known limitation of RLS, we demonstrate that class imbalance exacerbates this issue into a distinct spectral pathology: "tail" classes suffer from severe spectral collapse, rendering their subspaces numerically indistinguishable from noise. Standard Ridge Regression ($L_2$) fails to address this effectively as it applies isotropic regularization - a uniform penalty that is insufficient to stabilize the tail without over-shrinking the head. To address this, we propose Geometry-Spectral Rectification (GSR), a theoretically grounded framework that treats long-tailed learning as a spectral regularization problem. Unlike standard isotropic regularization (Ridge) which uniformly penalizes all eigenvalues, GSR acts as an anisotropic spectral filter, selectively inflating the collapsed eigenvalues of tail classes. We construct a structured, data-dependent spectral perturbation matrix $Δ$ that selectively inflates collapsed tail eigen-directions of the Gram matrix. Theoretical analysis proves that GSR guarantees an improved stable rank for the Gram matrix, ensuring numerical stability. Extensive experiments show that GSR establishes a new state-of-the-art for analytic CIL, offering a superior trade-off between computational efficiency and robust generalization in long-tailed settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。